microsoft / microsoft/SynapseML

Make Spark Serving Easier to Use

Open
#347 0 comments 0 reactions 1 assignee View on GitHub

@mhamilton723 is already working on this.

Since Jul 18, 2018.

enhancement
Dominant language
Scala
Stars
5.2k
Forks
868
Avg merge
22h 9m
Merged PRs (30d)
45

Description

- [ ] Add option to fail when it cannot parse the input
- [x] Add tests on ADB
- [x] Add syntactic sugars for adding input parsers
- [x] Add ability to reply with other types
- [ ] Fix offsets commited out of order exception

Repro:

```python
import mmlspark
from pyspark.sql.functions import length, col, lit, from_json
from pyspark.sql.types import *

df = spark.readStream.server() \
.address("localhost", 8888, "my_api") \
.load()

server = df.withColumn("reply", length(col("value"))) \
.withColumn("replyString", col("reply").cast(StringType()))\
.writeStream \
.server() \
.option("name", "my_api") \
.queryName("my_query") \
.option("replyCol", "replyString") \
.option("checkpointLocation", "checkpoints") \
.start()

import requests

r = requests.post(data="{foo:1}", url="http://localhost:8888/my_api")
print(r.text)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.