finos / finos/legend-studio

Feature request: Add more telemetry metrics and insight for usage analytics

Open
#2,017 0 comments 0 reactions 0 assignees View on GitHub
Application: Query Application: Studio Application: Taxonomy Component: UX/Design Studio Core Team Type: Feature Request Type: Mega-thread
Dominant language
TypeScript
Stars
111
Forks
149
Avg merge
9h 39m
Merged PRs (30d)
63

Description

### Similar requests

- [X] I have searched and found no existing similar requests

### How are you using Legend?

Legend Studio, Legend Query, Legend Taxonomy, Legend Pure IDE

### What problems are you trying to solve?

We have laid out the foundation for telemetry across all the apps, we now should systematically find metrics that worths capturing, they divide roughly into 2 camps:
- Usage metrics: what actions do users take, what do they use (potentially, for deeper study, how do they use that feature, where do they use it from, etc.)
- Performance metrics: For long-running actions, what is the timing, what is the load, etc.

I want the team to work on this thread together to exhaustively find all the interesting metrics to capture, also potentially, we can discuss popular capturing techniques to generalize that into some method inside of `ApplicationStore.TelemetryService`

And most importantly, after we capture those metrics, what is the best way to present those metrics in dashboard to provide value and insights into users' behavior

### Describe the solution you would like to see implemented

- [ ] Think about another kind of endpoint that does the absolute, call a bunch of APIs that pulls stats like: total number of queries, dataspaces, projects, etc. and create cron job to push these data to analytics service
- [ ] Include generic object to indicate users' workspace: `SDLC coordinates`, `GAV`, `Query ID`, etc.
- [ ] `Metrics` How can we track the number of times people flip between text/form mode - check for windows visibility
- [ ] Leaving/entering text-mode
- [ ] Service test runner
- [ ] Mapping test runner
- [ ] Service registration
- [ ] For compilation / graph load / execution
- [ ] Check entitlement
- [ ] Mapping algorithm
- [ ] Loading model
- [ ] Pushing change
- [ ] Updating workspace
- [ ] Updating project configuration
- [ ] Database builder
- [ ] Sample data generator: metrics for launch, with format info
- [ ] Query Builder: Preview data
- [ ] Query Builder: Typeahead
- [ ] `THINK` Consider the following aspects of metrics:
- [ ] For performance metrics, what kind of timings should we capture? For multi-staged timings, what do we do?
- [ ] How can we justify that a feature is popular, should we based it off the percentage of unique user using it?
- [ ] Think about how we can determine feature to deprecate
- [ ] Think about how we can make use of application session telemetry to study users' behavior per session, such as how frequently they do action a, b, c, etc.
- [ ] Think about the popularity of a feature, percentage of users that use the feature - just to see how popular the actions are

### Describe alternatives you have considered

_No response_

### Documentation, Design, Adoption, Migration Strategy

_No response_

### Contribution

- [ ] I would like to work on this feature

Contributor guide

Open the contributing guide

Research direction

Start with ApplicationStore.TelemetryService and inventory the existing telemetry foundation across Legend Studio, Legend Query, Legend Taxonomy, and Legend Pure IDE. Review the proposed analytics APIs, cron job, workspace identifiers, usage metrics, performance timings, and dashboard needs; done requires an agreed, implementable metrics and insight plan.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
analytics, observability-sre
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.