Skip to content
ArthurzWongPublic

About

event aggregator for tech and ai events in malaysia and singapore

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

8 Commits

Folders and files

Repository files navigation

SG/MY Tech + AI Event Aggregator

A local-first desk for finding, reviewing and exporting technology, startup and AI events across Singapore and Malaysia.

It collects event pages from curated sources, extracts event metadata, deduplicates the records, gives you a review desk to decide what is worth your time, and gets the keepers into your calendar or notes — as a downloadable .ics, a .csv, Google Calendar, or a Notion database.

Quick start

npm run dev            # start the review desk on http://127.0.0.1:4173

Open the desk and press Collect live to pull the latest events. Everything else — filters, notes, approvals, exports — works from the page.

npm run collect        # collect from the command line (same engine as the button)
npm run build          # copy data into public/ for static hosting
npm run export:ics     # write data/approved-events.ics and .csv for approved events

Sync to your own accounts (optional — copy .env.example to .env first):

npm run sync:calendar  # push approved events to Google Calendar
npm run sync:notion    # push approved events to a Notion database

What the desk does

Area What you get
Live collection Collect live runs every source server-side and reports which ones responded, all in one pass. No mock data — the bundled snapshot is real output from the same collector.
Review workflow Approve, reject, reset, star and annotate any event. Bulk-approve a filtered selection. Status, notes and stars persist to data/events.json through the API.
Filters Full-text search, status, timeframe (upcoming / next 7 days / next 30 days / past / no date), country, source and tag chips, plus five sort orders. Defaults to Upcoming so the desk opens on what is actually attendable.
Detail sheet Full description, organiser, venue, source, timestamps, and a notes box per event.
Export Download the approved set (or everything) as .ics or .csv straight from the header. Works on the static build too — the files are generated in the browser.
Sync Push approved events to Google Calendar or Notion from the header menu; already-synced events are skipped.
Sources Add or remove sources from the UI. Each source shows how many events it contributes.
Overview Live KPI strip: events tracked, upcoming in 30 days, approved, and new to review.

How it works

data/sources.json          sources to check (name, URL, country, tags, collector)
scripts/collector.mjs      fetch + parse + relevance filter + merge/dedupe
scripts/shared.mjs         env, ids, relevance scoring, ICS/CSV builders
scripts/sync.mjs           Google Calendar + Notion push
src/server.mjs             static host + JSON API + live collection + exports
public/                    the review desk (vanilla ES modules, no framework)
data/events.json           the local event database

Data flow. collector.mjs fetches each source, reads JSON-LD Event objects (or the Eventbrite API when a token is present), scores each one for tech/AI relevance, and merges it into data/events.json. Review decisions live on the same record, so a refresh never loses your notes or approvals — collector merges preserve status, notes, priority, starred and synced.

Offline / static mode. Opened without the Node server (for example on a static host), the desk loads the bundled public/events.json snapshot and keeps review decisions in localStorage. Exports still work; live collection, source editing and sync show a clear "needs the local server" message instead of failing silently.

API (served by npm run dev)

Method Path Purpose
GET /api/health Counts + server time
GET /api/events All events
PATCH /api/events/:id Update status, notes, priority, starred
DELETE /api/events/:id Remove an event
POST /api/collect Run a live collection pass
GET /api/sources · POST · DELETE /api/sources/:id Manage sources
GET /api/export/ics?scope=approved|all Calendar download
GET /api/export/csv?scope=approved|all Spreadsheet download
POST /api/sync/calendar · /api/sync/notion Push approved events

Keyboard shortcuts

/ focus search · c collect live · e open the export menu · Esc close a sheet.

Adding sources

Add a source in the UI (Sources → Add a source) or edit data/sources.json. A source works best when its page publishes application/ld+json Event data. JavaScript-heavy sites (several Eventbrite and GDG pages) return nothing to a plain fetch — they are listed but reported as skipped during collection rather than pretending to succeed. Set EVENTBRITE_TOKEN in .env to collect those via the official Eventbrite API.

Changing things

To change Edit
Sources data/sources.json or the Sources sheet
Relevance keywords, IDs, hashing scripts/shared.mjs
Parsing / collectors / merge rules scripts/collector.mjs
Colours and tokens public/styles.css (:root and [data-theme="light"])
Layout and copy public/index.html, public/app.js
API surface src/server.mjs

Deployment

Two hosting shapes are supported:

Static + serverless (Vercel). npm run build copies data/events.json and data/sources.json into public/, and the two functions in api/ expose GET /api/health and POST /api/collect. The desk detects which mode it is in:

Detected Behaviour
Full (local npm run dev) Review decisions persist to data/events.json; sources and sync are editable.
Hosted (serverless) Collect live fetches sources on demand and merges them into this device; review decisions, notes and stars are kept in localStorage. Source editing and sync ask for a local run.
Offline (plain static host) Bundled snapshot only; exports still work.

The /api/collect function is stateless by design — serverless filesystems are read-only, so it returns the freshly collected events to the browser rather than writing a database. Collection runs every source concurrently, so a full pass finishes in well under a second.

Node host. Deploy src/server.mjs to any Node platform for the full mode (persistent database, source editing, Google Calendar / Notion sync).

Scraping note

This project should respect robots.txt, site terms, rate limits and attribution. Prefer public APIs, RSS feeds or structured event pages. Collection is intentionally read-only and polite: one request per source per run, with a descriptive user agent.

About

event aggregator for tech and ai events in malaysia and singapore

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages