Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,10 +49,10 @@ Requires [Bun](https://bun.sh) at the release named in the `packageManager` fiel

```sh
bun install
bun run --filter geo-ip-service dev
bun run --filter api-service dev
```

`dev` restarts the service when a source file changes. Every service listens on `PORT`, default 3000, writes JSON log lines at `LOG_LEVEL`, default `info`, answers `GET /healthz`, and shuts down cleanly on `SIGTERM` or `SIGINT`. See [packages/service-kit](packages/service-kit/) for each of these. To run two services side by side, give each its own port:
`dev` restarts the service when a source file changes. A service that needs more configuration lists it in its own README, for example [geo-ip-service](apps/geo-ip-service/#running-locally). Every service listens on `PORT`, default 3000, writes JSON log lines at `LOG_LEVEL`, default `info`, answers `GET /healthz`, and shuts down cleanly on `SIGTERM` or `SIGINT`. See [packages/service-kit](packages/service-kit/) for each of these. To run two services side by side, give each its own port:

```sh
PORT=3001 bun run --filter api-service dev
Expand Down
128 changes: 128 additions & 0 deletions apps/geo-ip-service/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
# geo-ip-service

Resolves IP addresses to location and network data from the MaxMind GeoLite2 City and ASN databases. It is an internal service with no authentication, meant to be reached only by other services, never from the public edge. Services in this repository call it through [geo-ip-client](../../packages/geo-ip-client/).

## Running locally

The service reads the databases from a directory. To run it without a MaxMind license, copy the test databases under the names the service expects:

```sh
mkdir -p /tmp/geoip
cp apps/geo-ip-service/test-data/GeoLite2-City-Test.mmdb /tmp/geoip/GeoLite2-City.mmdb
cp apps/geo-ip-service/test-data/GeoLite2-ASN-Test.mmdb /tmp/geoip/GeoLite2-ASN.mmdb
GEOIP_DATA_DIR=/tmp/geoip bun run --filter geo-ip-service dev

curl localhost:3000/lookup/89.160.20.112
curl localhost:3000/lookup -H 'content-type: application/json' -d '{"addresses":["89.160.20.112","2001:480::1"],"h3Resolution":4}'
```

The test databases hold only a few made-up networks; [test-data](test-data/) says where to find the list.

## Configuration

| Variable | Default | Meaning |
| -------- | ------- | ------- |
| `GEOIP_DATA_DIR` | required | directory holding `GeoLite2-City.mmdb` and `GeoLite2-ASN.mmdb` |
| `MAX_BATCH_SIZE` | `10000` | most addresses accepted in one batch lookup, the same as the client's default batch size |
| `PORT`, `LOG_LEVEL` | | see [service-kit](../../packages/service-kit/#environment) |

## Databases

Both editions are loaded at startup. A file must hold the edition its name says, which is checked against the database's own metadata, so a City file in the ASN slot is refused.

- Without the City edition the service is `down`: `/healthz` answers `503` and lookups answer `503`.
- Without the ASN edition the service is `degraded`: lookups answer, and results have no `asNumber` or `asName`.
- When loading an edition fails and an earlier copy of it is loaded, the earlier copy keeps serving and the failure is reported in `/healthz` and `/metrics`.

## API

All bodies are JSON. Errors are `{ "error": "<message>" }`.

### `POST /lookup`

Looks up a batch of addresses.

```json
{ "addresses": ["89.160.20.112", "2001:480::1", "10.0.0.1", "not-an-address"], "h3Resolution": 4 }
```

`addresses` is required. `h3Resolution` is optional, an integer from 0 to 15; when given, results with coordinates also carry the H3 cell holding them at that resolution and the cell's center.

The answer has one result per requested address, in the same order, duplicates included:

```json
{
"results": [
{
"address": "89.160.20.112",
"status": "found",
"location": {
"coordinates": { "latitude": 58.4167, "longitude": 15.6167 },
"countryCode": "SE",
"countryName": "Sweden",
"regionName": "Östergötland County",
"cityName": "Linköping",
"timeZone": "Europe/Stockholm",
"asNumber": 29518,
"asName": "Bredband2 AB",
"h3": { "resolution": 4, "cell": "8408b3dffffffff", "center": { "latitude": 58.40578421882332, "longitude": 15.778067337833107 } }
}
},
{ "address": "10.0.0.1", "status": "not-found" },
{ "address": "not-an-address", "status": "invalid" }
]
}
```

| Status | Meaning |
| ------ | ------- |
| `found` | at least one edition has a record for the address; `location` holds what the records have |
| `not-found` | a valid IPv4 or IPv6 address that neither edition has a record for |
| `invalid` | not an IPv4 or IPv6 address |

Every `location` field is optional and present only when the databases have it:

| Field | Source |
| ----- | ------ |
| `coordinates` | City: latitude and longitude |
| `countryCode`, `countryName` | City: country ISO code and English name |
| `regionName` | City: English name of the first subdivision |
| `cityName` | City: English city name |
| `timeZone` | City: time zone |
| `asNumber`, `asName` | ASN: autonomous system number and organization |
| `h3` | computed from `coordinates` when `h3Resolution` is given |

An IPv4-mapped IPv6 address such as `::ffff:89.160.20.112` gives the same result as the IPv4 address.

| HTTP status | When |
| ----------- | ---- |
| `200` | the batch was answered, whatever the status of each address |
| `400` | the body is not JSON, `addresses` is missing or holds something other than strings, or `h3Resolution` is out of range |
| `413` | `addresses` holds more than `MAX_BATCH_SIZE` entries |
| `503` | the City edition is not loaded |

### `GET /lookup/:address`

Looks up one address, with `h3Resolution` as an optional query parameter: `GET /lookup/89.160.20.112?h3Resolution=4`. The body is a single result in the shape above. The HTTP status follows the result: `200` for `found`, `404` for `not-found`, `400` for `invalid` or a bad `h3Resolution`, and `503` when the City edition is not loaded. An IPv6 address can be written as is or percent-encoded.

### `GET /healthz`

The [service-kit health report](../../packages/service-kit/#health) with one check per edition, holding its `buildDate`, its `loadedAt` time and the `lastError` of its last load, if that failed.

### `GET /metrics`

Metrics in the Prometheus text format:

| Metric | Type | Labels | Meaning |
| ------ | ---- | ------ | ------- |
| `geoip_database_loaded` | gauge | `edition` | `1` when the edition is loaded |
| `geoip_database_load_failed` | gauge | `edition` | `1` when the last attempt to load the edition failed |
| `geoip_database_build_timestamp_seconds` | gauge | `edition` | build time of the loaded edition, for alerting on its age |
| `geoip_database_load_timestamp_seconds` | gauge | `edition` | time the loaded edition was loaded |
| `geoip_lookups_total` | counter | `status` | addresses looked up, by result |
| `geoip_requests_total` | counter | `route`, `code` | lookup requests by route (`batch` or `single`) and HTTP status |
| `geoip_batch_duration_seconds` | summary | | time to answer a batch, from receiving it to the encoded answer |

## Tests

The tests run against MaxMind's test databases in [test-data](test-data/), so they need no license key or network access.
5 changes: 4 additions & 1 deletion apps/geo-ip-service/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@
"start": "bun src/main.ts"
},
"dependencies": {
"@anyone-protocol/service-kit": "workspace:*"
"@anyone-protocol/geo-ip-client": "workspace:*",
"@anyone-protocol/service-kit": "workspace:*",
"h3-js": "4.5.0",
"maxmind": "5.0.7"
}
}
215 changes: 204 additions & 11 deletions apps/geo-ip-service/src/app.test.ts
Original file line number Diff line number Diff line change
@@ -1,29 +1,222 @@
import { afterAll, beforeAll, describe, expect, it } from 'bun:test';
import { GeoIpClient, type GeoLocation } from '@anyone-protocol/geo-ip-client';
import { afterEach, describe, expect, it } from 'bun:test';
import { join } from 'node:path';
import { startServer } from './app';
import { ASN, CITY, type Edition, GeoDatabases } from './databases';

let server: ReturnType<typeof startServer>;
const TEST_DATA = join(import.meta.dir, '..', 'test-data');
const TEST_FILES: Record<Edition, string> = {
[CITY]: join(TEST_DATA, 'GeoLite2-City-Test.mmdb'),
[ASN]: join(TEST_DATA, 'GeoLite2-ASN-Test.mmdb'),
};

beforeAll(() => {
server = startServer(0);
const LINKOPING: GeoLocation = {
coordinates: { latitude: 58.4167, longitude: 15.6167 },
countryCode: 'SE',
countryName: 'Sweden',
regionName: 'Östergötland County',
cityName: 'Linköping',
timeZone: 'Europe/Stockholm',
asNumber: 29518,
asName: 'Bredband2 AB',
};

const LINKOPING_H3 = {
resolution: 4,
cell: '8408b3dffffffff',
center: { latitude: 58.40578421882332, longitude: 15.778067337833107 },
};

const SAN_DIEGO: GeoLocation = {
coordinates: { latitude: 32.7203, longitude: -117.1552 },
countryCode: 'US',
countryName: 'United States',
regionName: 'California',
cityName: 'San Diego',
timeZone: 'America/Los_Angeles',
};

const TELSTRA: GeoLocation = { asNumber: 1221, asName: 'Telstra Pty Ltd' };

let server: ReturnType<typeof startServer> | undefined;

async function serve({ editions = [CITY, ASN], maxBatchSize = 10 }: { editions?: Edition[]; maxBatchSize?: number } = {}) {
const databases = new GeoDatabases();
for (const edition of editions) {
await databases.load(edition, TEST_FILES[edition]);
}
server = startServer({ port: 0, databases, maxBatchSize });
return server;
}

function post(body: unknown) {
return fetch(new URL('/lookup', server?.url), {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: typeof body === 'string' ? body : JSON.stringify(body),
});
}

function get(path: string) {
return fetch(new URL(path, server?.url));
}

afterEach(async () => {
await server?.stop(true);
server = undefined;
});

describe('POST /lookup', () => {
it('answers every address in order, reporting unknown and invalid ones per item', async () => {
await serve();

const response = await post({ addresses: ['89.160.20.112', '2001:480::1', '1.128.0.1', '10.0.0.1', 'not-an-address', '89.160.20.112'] });

expect(response.status).toBe(200);
expect(await response.json()).toEqual({
results: [
{ address: '89.160.20.112', status: 'found', location: LINKOPING },
{ address: '2001:480::1', status: 'found', location: SAN_DIEGO },
{ address: '1.128.0.1', status: 'found', location: TELSTRA },
{ address: '10.0.0.1', status: 'not-found' },
{ address: 'not-an-address', status: 'invalid' },
{ address: '89.160.20.112', status: 'found', location: LINKOPING },
],
});
});

it('adds the H3 cell and its center at the requested resolution to addresses with coordinates', async () => {
await serve();

const response = await post({ addresses: ['89.160.20.112', '1.128.0.1'], h3Resolution: 4 });

expect(await response.json()).toEqual({
results: [
{ address: '89.160.20.112', status: 'found', location: { ...LINKOPING, h3: LINKOPING_H3 } },
{ address: '1.128.0.1', status: 'found', location: TELSTRA },
],
});
});

it.each([
['a body that is not JSON', '{"addresses": [', 400],
['a body without addresses', { h3Resolution: 4 }, 400],
['addresses that are not all strings', { addresses: ['89.160.20.112', 7] }, 400],
['an H3 resolution above 15', { addresses: ['89.160.20.112'], h3Resolution: 16 }, 400],
['an H3 resolution that is not an integer', { addresses: ['89.160.20.112'], h3Resolution: 4.5 }, 400],
['more addresses than the batch limit', { addresses: Array.from({ length: 11 }, () => '89.160.20.112') }, 413],
])('rejects %s', async (_, body, status) => {
await serve();

const response = await post(body);

expect(response.status).toBe(status);
expect(await response.json()).toEqual({ error: expect.any(String) });
});
});

afterAll(async () => {
await server.stop();
describe('GET /lookup/:address', () => {
it.each([
['an IPv4 address', '89.160.20.112', 200, { status: 'found', location: LINKOPING }],
['an IPv6 address', '2001:480::1', 200, { status: 'found', location: SAN_DIEGO }],
['an encoded IPv6 address', encodeURIComponent('2001:480::1'), 200, { status: 'found', location: SAN_DIEGO }],
['an address in no database', '10.0.0.1', 404, { status: 'not-found' }],
['something that is not an address', 'not-an-address', 400, { status: 'invalid' }],
])('answers %s', async (_, path, status, body) => {
await serve();

const response = await get(`/lookup/${path}`);

expect(response.status).toBe(status);
expect(await response.json()).toEqual({ address: decodeURIComponent(path), ...body });
});

it('adds the H3 cell when a resolution is given', async () => {
await serve();

expect(await (await get('/lookup/89.160.20.112?h3Resolution=4')).json()).toMatchObject({ location: { h3: LINKOPING_H3 } });
expect((await get('/lookup/89.160.20.112?h3Resolution=x')).status).toBe(400);
});
});

describe('geo-ip-client', () => {
it('reads the same results the service answers', async () => {
await serve();
const client = new GeoIpClient({ baseUrl: server!.url.href, h3Resolution: 4 });

const results = await client.lookupMany(['89.160.20.112', '10.0.0.1', 'not-an-address']);

expect(Object.fromEntries(results)).toEqual({
'89.160.20.112': { status: 'found', location: { ...LINKOPING, h3: LINKOPING_H3 }, stale: false },
'10.0.0.1': { status: 'not-found', stale: false },
'not-an-address': { status: 'invalid', stale: false },
});
});
});

describe('GET /healthz', () => {
it('reports the service as ok', async () => {
const response = await fetch(new URL('/healthz', server.url));
it('reports the build date and load time of each database', async () => {
await serve();

const response = await get('/healthz');

expect(response.status).toBe(200);
expect(await response.json()).toMatchObject({ service: 'geo-ip-service', status: 'ok' });
expect(await response.json()).toMatchObject({
service: 'geo-ip-service',
status: 'ok',
checks: {
[CITY]: { status: 'ok', buildDate: '2026-10-14T19:30:47.000Z', loadedAt: expect.any(String), lastError: null },
[ASN]: { status: 'ok', buildDate: '2026-10-14T19:30:47.000Z', loadedAt: expect.any(String), lastError: null },
},
});
});

it('is degraded without the ASN database and still answers lookups', async () => {
await serve({ editions: [CITY] });

expect(await (await get('/healthz')).json()).toMatchObject({ status: 'degraded', checks: { [ASN]: { status: 'degraded' } } });
expect((await get('/lookup/89.160.20.112')).status).toBe(200);
});

it('is down without the City database, and lookups are unavailable', async () => {
await serve({ editions: [ASN] });

expect((await get('/healthz')).status).toBe(503);
expect((await get('/lookup/89.160.20.112')).status).toBe(503);
expect((await post({ addresses: ['89.160.20.112'] })).status).toBe(503);
});
});

describe('GET /metrics', () => {
it('exposes database ages and lookup counts in the Prometheus text format', async () => {
await serve({ editions: [CITY] });
await post({ addresses: ['89.160.20.112', '10.0.0.1', 'not-an-address'] });
await post({ addresses: 'none' });
await get('/lookup/89.160.20.112');

const response = await get('/metrics');
const text = await response.text();

expect(response.headers.get('content-type')).toBe('text/plain; version=0.0.4; charset=utf-8');
expect(text).toContain('geoip_database_loaded{edition="GeoLite2-City"} 1');
expect(text).toContain('geoip_database_loaded{edition="GeoLite2-ASN"} 0');
expect(text).toContain('geoip_database_load_failed{edition="GeoLite2-ASN"} 0');
expect(text).toContain(`geoip_database_build_timestamp_seconds{edition="GeoLite2-City"} ${Date.parse('2026-10-14T19:30:47Z') / 1000}`);
expect(text).toContain('geoip_lookups_total{status="found"} 2');
expect(text).toContain('geoip_lookups_total{status="not-found"} 1');
expect(text).toContain('geoip_lookups_total{status="invalid"} 1');
expect(text).toContain('geoip_requests_total{route="batch",code="200"} 1');
expect(text).toContain('geoip_requests_total{route="batch",code="400"} 1');
expect(text).toContain('geoip_requests_total{route="single",code="200"} 1');
expect(text).toContain('geoip_batch_duration_seconds_count 1');
expect(text.endsWith('\n')).toBe(true);
});
});

describe('an unknown path', () => {
it('is not found', async () => {
const response = await fetch(new URL('/missing', server.url));
await serve();

expect(response.status).toBe(404);
expect((await get('/missing')).status).toBe(404);
});
});
Loading
Loading