Azure / Azure/azure-kusto-node

Progressive mode in azure-kusto-data

Open
#225 3 comments 0 reactions 1 assignee Claimed by @AsafMah View on GitHub
enhancement
Dominant language
TypeScript
Stars
54
Forks
46
PR merge metrics
No merged PRs in 30d

Description

Hi folks,

I have a two part question: in looking at the [C#](https://github.com/Azure/azure-kusto-samples-dotnet/blob/master/client/HelloKustoV2/Program.cs) and [Rust](https://github.com/Azure/azure-kusto-rust/blob/main/azure-kusto-data/examples/query.rs) examples, and the [Query v2 REST API doc](https://learn.microsoft.com/en-us/azure/data-explorer/kusto/api/rest/response2), it looks like Kusto supports a streaming model called "progressive" mode. First, is this the correct understanding of these APIs?

Second, does the azure-kusto-data Node SDK support progressive mode? Playing around my own code and a large data set, the SDK seems to wait until all the results are downloaded, rather than streaming the results. Passing `results_progressive_enabled` as a request option in a similar way to the C# example seems to result in errors.

Here's some sample code, more or less lifted from the example, to demonstrate:

```TypeScript
import {
Client as KustoClient,
KustoConnectionStringBuilder,
ClientRequestProperties as KustoClientRequestProperties,
} from "azure-kusto-data";

const clusterName = "";
const databaseName = "";
const queryStr = "MyTable | where value == 10";

const connStr = KustoConnectionStringBuilder.withAzLoginIdentity(
`https://${clusterName}.kusto.windows.net`);
const client = new KustoClient(connStr);

const requestProperties = new KustoClientRequestProperties();
requestProperties.setOption("results_progressive_enabled", true);

// error happens here
const response = await client.executeQuery(databaseName, queryStr, requestProperties);
```

The produced output is very long, so I just copied the initial error code.

```Output
{
code: "Z_DATA_ERROR",
errno: -3,
message: "incorrect header check",

....
```

This code does work when I omit the `results_progressive_enabled` request property.

Thanks,

Will Gries

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.