microsoft / microsoft/msticpy

[Bug]: RiskIQ TI Provider does not seem functional anymore and docs for config and API access are out of date

Open
#820 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement good first issue
Dominant language
Python
Stars
2k
Forks
343
PR merge metrics
No merged PRs in 30d

Description

Describe the bug

RiskIQ was acquired by Microsoft. At some point api.passivetotal.org might have stopped working. When attempting to use a TI lookup with this provider, I get a TLS error for using that endpoint.

To Reproduce

Config

Azure:
  auth_methods:
  - cli
  - devicecode
  cloud: global
TIProviders:
  RiskIQ:
    Primary: true
    Provider: RiskIQ

Code

import numpy as np  # Silent undocumented dependency that causes warnings if not loaded
import msticpy as mp

mp.init_notebook()
ti = mp.TILookup()
ti.lookup_item(item='20.54.231.124', providers=['RiskIQ'])

Expected behavior

I expected to get at least a response that I'm not authorized to use the API and require credentials.

Screenshots and/or Traceback

2025-01-30 17:49:00,860: WARNING - Certificate did not match expected hostname: api.passivetotal.org. Certificate: {'subject': ((('countryName', 'US'),), (('stateOrProvinceName', 'WA'),), (('localityName', 'Redmond'),), (('organizationName', 'Microsoft Corporation'),), (('commonName', '*.azureedge.net'),)), 'issuer': ((('countryName', 'US'),), (('organizationName', 'Microsoft Corporation'),), (('commonName', 'Microsoft Azure RSA TLS Issuing CA 04'),)), 'version': 3, 'serialNumber': '330182F0C508084CF4688786E300000182F0C5', 'notBefore': 'Jan 18 14:39:25 2025 GMT', 'notAfter': 'Jul 17 14:39:25 2025 GMT', 'subjectAltName': (('DNS', '*.azureedge.net'), ('DNS', '*.media.microsoftstream.com'), ('DNS', '*.origin.mediaservices.windows.net'), ('DNS', '*.streaming.mediaservices.windows.net')), 'OCSP': ('http://oneocsp.microsoft.com/ocsp',), 'caIssuers': ('http://www.microsoft.com/pkiops/certs/Microsoft%20Azure%20RSA%20TLS%20Issuing%20CA%2004%20-%20xsign.crt',), 'crlDistributionPoints': ('http://www.microsoft.com/pkiops/crl/Microsoft%20Azure%20RSA%20TLS%20Issuing%20CA%2004.crl',)} (connection#547)

Traceback

Observables processed: 0%
0/1 [00:00<?, ?obs/s]
2025-01-30 17:49:00,860: WARNING - Certificate did not match expected hostname: api.passivetotal.org. Certificate: {'subject': ((('countryName', 'US'),), (('stateOrProvinceName', 'WA'),), (('localityName', 'Redmond'),), (('organizationName', 'Microsoft Corporation'),), (('commonName', '*.azureedge.net'),)), 'issuer': ((('countryName', 'US'),), (('organizationName', 'Microsoft Corporation'),), (('commonName', 'Microsoft Azure RSA TLS Issuing CA 04'),)), 'version': 3, 'serialNumber': '330182F0C508084CF4688786E300000182F0C5', 'notBefore': 'Jan 18 14:39:25 2025 GMT', 'notAfter': 'Jul 17 14:39:25 2025 GMT', 'subjectAltName': (('DNS', '*.azureedge.net'), ('DNS', '*.media.microsoftstream.com'), ('DNS', '*.origin.mediaservices.windows.net'), ('DNS', '*.streaming.mediaservices.windows.net')), 'OCSP': ('http://oneocsp.microsoft.com/ocsp',), 'caIssuers': ('http://www.microsoft.com/pkiops/certs/Microsoft%20Azure%20RSA%20TLS%20Issuing%20CA%2004%20-%20xsign.crt',), 'crlDistributionPoints': ('http://www.microsoft.com/pkiops/crl/Microsoft%20Azure%20RSA%20TLS%20Issuing%20CA%2004.crl',)} (connection#547)
---------------------------------------------------------------------------
CertificateError                          Traceback (most recent call last)
File ~/.venv/secnb/lib/python3.11/site-packages/urllib3/connectionpool.py:714, in HTTPConnectionPool.urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
...
SSLError: HTTPSConnectionPool(host='api.passivetotal.org', port=443): Max retries exceeded with url: /v2/cards/summary?query=20.54.231.124 (Caused by SSLError(CertificateError("hostname 'api.passivetotal.org' doesn't match either of '*.azureedge.net', '*.media.microsoftstream.com', '*.origin.mediaservices.windows.net', '*.streaming.mediaservices.windows.net'")))

Environment (please complete the following information):

  • Python Version: 3.11
  • OS: Ubuntu
  • Python environment: Python virtual env
  • MSTICPy Version: 2.15.0

Additional context

It's somewhat unclear now how to go about and get credentials for Risk IQ API access to enable this provider.

See: https://learn.microsoft.com/en-us/connectors/riskiqpassivetotal/#how-to-get-credentials

Register for a test API key at RiskIQ Security Intelligence Services or contact your account representative (support@riskiq.com) to identify your existing customer keys.

RiskIQ Security Intelligence Services link is broken. https://api.riskiq.net/api/concepts.html doesn't seem to be a valid site anymore.

These docs and any others I tried to find seem out of date and there's no up to date info on how to get authenticated to and make use of RiskIQ with msticpy.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the RiskIQ provider used by msticpy.TILookup and verify whether api.passivetotal.org and the linked RiskIQ credential guidance are still valid. Done would require a confirmed current authentication and API path, plus updated documentation or a clear indication that the provider is no longer supported.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, documentation
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
22/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.