microsoft / microsoft/sql-server-samples

conducting regression analysis using R via SQL 2017

Open
#398 0 comments 0 reactions 1 assignee View on GitHub

Nobody has claimed this yet.

Dominant language
PowerShell
Stars
11.2k
Forks
9.1k
Avg merge
2d 7h
Merged PRs (30d)
14

Description

I want perform regression analysis using R code via SQL Server 2017 (it's integrated here).

Here is the native R code working with the csv

The main matter of code that we perform regression separately by groups [CustomerName]+[ItemRelation]+[DocumentNum]+[DocumentYear]

df=read.csv("C:/Users/synthex/Desktop/re.csv", sep=";",dec=",")

#load needed library

library(tidyverse)
library(broom)
#order dataset
df=df[ order(df[,5]),]
df=df[ order(df[,6]),]
#delete signs
df$Customer<-gsub("\\-","",df$Customer)

#create lm function for separately by group regression

my_lm <- function(df) {
  lm(SaleCount~IsPromo, data = df)
}

reg=df %>%
  group_by(CustomerName,ItemRelation,DocumentNum,DocumentYear) %>%
  nest() %>%
  mutate(fit = map(data, my_lm),
         tidy = map(fit, tidy)) %>%
  select(-fit, - data) %>%
  unnest()

w=aggregate(df$action, by=list(CustomerName=df$CustomerName,ItemRelation=df$ItemRelation, DocumentNum=df$DocumentNum, DocumentYear=df$DocumentYear), FUN=sum)
View(w)

# multiply each group by the number of days of the action
EA<-data.frame(reg$CustomerName,reg$ItemRelation,reg$DocumentNum,reg$DocumentYear, reg$estimate*w$x)

#del intercepts
toDelete <- seq(2, nrow(EA), 2)
newdat=EA[ toDelete ,]
View(newdat)

The finished result: this code runs in SSMS

So what I did:

EXECUTE sp_execute_external_script
      @language = N'R'
    , @script = N' OutputDataSet <- InputDataSet;'
    , @input_data_1 = N' SELECT     [CustomerName]
      ,[ItemRelation]
      ,[SaleCount]
      ,[DocumentNum]
      ,[DocumentYear]
      ,[IsPromo]
 FROM [Action].[dbo].[promo_data];'

    WITH RESULT SETS (([CustomerName] nvarchar(max) NOT NULL, [ItemRelation] int NOT NULL,
     [SaleCount] int NOT NULL,[DocumentNum]  int NOT NULL,
    [DocumentYear] int NOT NULL, [IsPromo] int NOT NULL));


df=as.data.frame(InputDataSet)

Message 102, level 15, state 1, line 17
Incorrect syntax near the "=" construct.

So, how perform regression analysis in SQL separately by groups?

Note, all coefficients must be saved, because new data come to the sql, should already automatically calculate by the equation of constructed model for each group.

The above code simply estimates the impact of the action, the beta coefficients of each group multiplies by the number of days of the action for each group.

If it is needed, here is a reproducible example:

df=structure(list(CustomerName = structure(c(1L, 2L, 3L, 3L, 1L, 
2L, 3L, 3L, 4L, 4L, 4L, 1L, 2L, 3L, 3L, 4L, 4L, 4L, 4L, 4L, 4L, 
4L, 1L, 2L, 3L, 3L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 
1L, 2L, 3L, 3L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 1L, 
2L, 3L, 3L, 4L, 4L, 4L, 4L, 4L), .Label = c("Attacks of the vehicle", 
"Auchan TS", "Tape of the vehicle", "X5 Retail Group"), class = "factor"), 
    ItemRelation = c(13322L, 13322L, 158121L, 158122L, 13322L, 
    13322L, 158121L, 158122L, 11592L, 13189L, 13191L, 13322L, 
    13322L, 158121L, 158122L, 11592L, 13189L, 13191L, 158121L, 
    158121L, 158122L, 158122L, 13322L, 13322L, 158121L, 158122L, 
    11592L, 13189L, 13191L, 157186L, 157192L, 158009L, 158010L, 
    158121L, 158121L, 158122L, 158122L, 13322L, 13322L, 158121L, 
    158122L, 11592L, 13189L, 13191L, 157186L, 157192L, 158009L, 
    158010L, 158121L, 158121L, 158122L, 158122L, 13322L, 13322L, 
    158121L, 158122L, 11514L, 11592L, 11623L, 13189L, 13191L), 
    SaleCount = c(10L, 35L, 340L, 260L, 3L, 31L, 420L, 380L, 
    45L, 135L, 852L, 1L, 34L, 360L, 140L, 14L, 62L, 501L, 0L, 
    560L, 640L, 0L, 0L, 16L, 0L, 0L, 15L, 66L, 542L, 49L, 228L, 
    3360L, 5720L, 980L, 0L, 0L, 1280L, 9L, 29L, 200L, 120L, 46L, 
    68L, 569L, 52L, 250L, 2360L, 3140L, 1640L, 0L, 0L, 1820L, 
    5L, 33L, 260L, 220L, 665L, 25L, -10L, 62L, 281L), DocumentNum = c(36L, 
    4L, 41L, 41L, 36L, 4L, 41L, 41L, 33L, 33L, 33L, 36L, 4L, 
    41L, 41L, 33L, 33L, 33L, 63L, 62L, 62L, 63L, 36L, 4L, 41L, 
    41L, 33L, 33L, 33L, 57L, 56L, 12L, 12L, 62L, 63L, 63L, 62L, 
    36L, 4L, 41L, 41L, 33L, 33L, 33L, 57L, 56L, 12L, 12L, 62L, 
    63L, 63L, 62L, 36L, 4L, 41L, 41L, 60L, 33L, 71L, 33L, 33L
    ), DocumentYear = c(2017L, 2017L, 2017L, 2017L, 2017L, 2017L, 
    2017L, 2017L, 2017L, 2017L, 2017L, 2017L, 2017L, 2017L, 2017L, 
    2017L, 2017L, 2017L, 2017L, 2017L, 2017L, 2017L, 2017L, 2017L, 
    2017L, 2017L, 2017L, 2017L, 2017L, 2017L, 2017L, 2017L, 2017L, 
    2017L, 2017L, 2017L, 2017L, 2017L, 2017L, 2017L, 2017L, 2017L, 
    2017L, 2017L, 2017L, 2017L, 2017L, 2017L, 2017L, 2017L, 2017L, 
    2017L, 2017L, 2017L, 2017L, 2017L, 2017L, 2017L, 2017L, 2017L, 
    2017L), IsPromo = c(0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 
    0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 
    0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 
    0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 
    0L, 0L, 0L, 0L, 0L, 0L)), .Names = c("CustomerName", "ItemRelation", 
"SaleCount", "DocumentNum", "DocumentYear", "IsPromo"), class = "data.frame", row.names = c(NA, 
-61L))

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.