JuliaWeb / JuliaWeb/GeoIP.jl

Package restructuring

Open
#41 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Julia
Stars
26
Forks
11
PR merge metrics
No merged PRs in 30d

Description

I want to summarize here problems that I see with current implementation and some ideas how to overcome it.

# Behind the scenes data downloading

In current implementation, data is loaded invisibly for the user. Moreover, it is not only loaded invisibly, it also downloads invisibly.

It leads to the following issues:

1. Unpredictable times of first `geolocate` call, it can take from milliseconds (actual lookup) to seconds or even minutes (when data is loaded).
2. Uncontrollable behaviour: user can't choose whether he wants to load data from an existing file, whether he wants to update the database, or even which base to use.
3. It is hard to switch from IPv4 to IPv6, an application should load both bases behind the scene and then somehow choose which one to use. See #21
4. It is hard to change localization, once again, all necessary files should be loaded behind the scene. It generates additional memory pressure. See #22
5. It is hard to switch from CSV to MaxMindDB, because it is not quite clear which base to use. See #26

Solution to all of these problems is the following methods which are accessible by user:

1. `load`: it should accept various parameters and modes. User can choose between local and internet data loading, between different database formats and localization
2. `update!`: it should accept parameters similar to `load, but it should validate the current state of the database and update database if new version is available.
3. `geolocate` should be changed to `geolocate(::DB, ::IP)`. For convenience, `getindex` method can be added `db[IP]` which works as `geolocate`.

# Loaded Data structure and results

In the current implementation `DataFrame` is used as a storage format, and `Dict{String, Any}` used as a return query format.

It leads to the following issues

1. `DataFrame` is type unstable by construction, so improper use can lead to unnecessary allocations and overall slowness.
2. `Row` construction is rather slow
3. Output is type unstable, making it the reason of slowdown in final application.

Possible solution:

1. Use `StructArray` or `Vector` of `GeoResult` structs.
2. Return `GeoResult`, which should be concretely typed and have a fixed number of fields. Use sentinel values instead of missing data.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.