combust / combust/mleap

Integer features being converted to double in Mleap

Open
#737 2 comments 1 reaction 0 assignees View on GitHub
Dominant language
Scala
Stars
1.5k
Forks
315
PR merge metrics
No merged PRs in 30d

Description

I've trained a Logistic Regression model with Pyspark, some of these are integers. When the model is loaded into a serving container the schema for the model has those features as floats. According to https://mleap-docs.combust.ml/core-concepts/data-frames/data-types.html Integers are supported

### code for saving the model ####
def save_model(self, model, predictions):
"""
Save model to a GCS bucket for use in inference
"""
model_file = f"{self.model_name}.zip"
tmp_location = f"/tmp/models/{model_file}"
model.serializeToBundle(f"jar:file:{tmp_location}", predictions)

### the schema of the predictions passed with the model ###
root
|-- client_id: string (nullable = true)
|-- dispatch_country: string (nullable = true)
|-- dw_classname: string (nullable = true)
|-- language_detected: string (nullable = true)
|-- photos_text_present: boolean (nullable = true)
|-- platform_id: string (nullable = true)
|-- site_group: string (nullable = true)
|-- territory: string (nullable = true)
|-- actioned: boolean (nullable = true)
|-- bcp_brands__count: integer (nullable = true)
|-- bcp_collections__count: integer (nullable = true)
|-- bcp_products__count: integer (nullable = true)
|-- photos_matched_logo_types__count: integer (nullable = true)
|-- image_match_types__count: integer (nullable = true)
|-- photos_matched_friendly_id__count: integer (nullable = true)

## The code for loading the saved model

docker pull combustml/mleap-spring-boot:0.17.0-SNAPSHOT
docker run -p 8080:8080 -v /tmp/models:/models -e SERVER_PORT=8080 combustml/mleap-spring-boot:0.17.0-SNAPSHOT

body='{"modelName":"enforcement_structured","uri":"file:/models/enforcement_structured.zip","config":{"memoryTimeout":900000,"diskTimeout":900000},"force":false}'
curl --header "Content-Type: application/json" --request POST --data "$body" http://localhost:8080/models

### The schema of calling the meta endpoint ###
curl --header "Content-Type: application/json" --request GET http://localhost:8080/models/enforcement_structured/meta

```yaml
{"bundle":{"uid":"b837d936-7d86-4c94-86ad-d215b7285297","name":"PipelineModel_aa1fc762aa64","format":"JSON","version":"0.16.0","timestamp":"2021-01-27T12:42:19.363"},

"inputSchema":{"fields":[
{"name":"site_group","dataType":{"base":"STRING","shape":{"base":"SCALAR","isNullable":true}}},

{"name":"language_detected","dataType":{"base":"STRING","shape":{"base":"SCALAR","isNullable":true}}},

{"name":"image_match_types__count","dataType":{"base":"DOUBLE","shape":{"base":"SCALAR","isNullable":true}}},

{"name":"client_id","dataType":{"base":"STRING","shape":{"base":"SCALAR","isNullable":true}}},

{"name":"bcp_brands__count","dataType":{"base":"DOUBLE","shape":{"base":"SCALAR","isNullable":true}}},

{"name":"photos_text_present","dataType":{"base":"DOUBLE","shape":{"base":"SCALAR","isNullable":true}}},

{"name":"dispatch_country","dataType":{"base":"STRING","shape":{"base":"SCALAR","isNullable":true}}},

{"name":"bcp_products__count","dataType":{"base":"DOUBLE","shape":{"base":"SCALAR","isNullable":true}}},

{"name":"photos_matched_friendly_id__count","dataType":{"base":"DOUBLE","shape":{"base":"SCALAR","isNullable":true}}},

{"name":"territory","dataType":{"base":"STRING","shape":{"base":"SCALAR","isNullable":true}}},

{"name":"bcp_collections__count","dataType":{"base":"DOUBLE","shape":{"base":"SCALAR","isNullable":true}}},

{"name":"dw_classname","dataType":{"base":"STRING","shape":{"base":"SCALAR","isNullable":true}}},

{"name":"photos_matched_logo_types__count","dataType":{"base":"DOUBLE","shape":{"base":"SCALAR","isNullable":true}}},

{"name":"platform_id","dataType":{"base":"STRING","shape":{"base":"SCALAR","isNullable":true}}}]},

"outputSchema":{"fields":[{"name":"platform_id_indexed","dataType":{"base":"DOUBLE","shape":{"base":"SCALAR","isNullable":false}}},{"name":"features","dataType":{"base":"DOUBLE","shape":{"base":"TENSOR","isNullable":true,"tensorShape":{"dimensions":[{"size":14,"name":""}]}}}},{"name":"territory_indexed","dataType":{"base":"DOUBLE","shape":{"base":"SCALAR","isNullable":false}}},{"name":"prediction","dataType":{"base":"DOUBLE","shape":{"base":"SCALAR","isNullable":false}}},{"name":"client_id_indexed","dataType":{"base":"DOUBLE","shape":{"base":"SCALAR","isNullable":false}}},{"name":"language_detected_indexed","dataType":{"base":"DOUBLE","shape":{"base":"SCALAR","isNullable":false}}},{"name":"site_group_indexed","dataType":{"base":"DOUBLE","shape":{"base":"SCALAR","isNullable":false}}},{"name":"dispatch_country_indexed","dataType":{"base":"DOUBLE","shape":{"base":"SCALAR","isNullable":false}}},{"name":"dw_classname_indexed","dataType":{"base":"DOUBLE","shape":{"base":"SCALAR","isNullable":false}}},{"name":"rawPrediction","dataType":{"base":"DOUBLE","shape":{"base":"TENSOR","isNullable":true,"tensorShape":{"dimensions":[{"size":2,"name":""}]}}}},{"name":"probability","dataType":{"base":"DOUBLE","shape":{"base":"TENSOR","isNullable":true,"tensorShape":{"dimensions":[{"size":2,"name":""}]}}}}]}}
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.