Tom Wier

Software engineer · Community Health Toolkit

I work on the path health data takes after it leaves a phone with no signal.

The Community Health Toolkit (CHT) is open-source software for community health workers. I work on three parts of it: the core framework that keeps the data moving, the pipeline that turns it into analytics, and the bridge to other health systems.

How the three projects connect

Data flow across the three CHT projects A health worker's phone replicates to the CHT server. From there, CHT Sync carries data through couch2pg, PostgreSQL and dbt to dashboards, and CHT Interoperability exchanges data through an OpenHIM mediator and a FHIR server with OpenMRS. CHT Core PhonePouchDB, offline CHT serverAPI, CouchDB replicates CHT Sync couch2pgtails _changes PostgreSQLraw JSONB dbtanalytics models Dashboardsprogram teams CHT Interoperability Mediatorthrough OpenHIM HAPI FHIRshared store OpenMRSfacility records
  1. CHT Core Works offline on the phone, then replicates to the server when there's a connection.
  2. CHT Sync Copies server data into PostgreSQL, where dbt shapes it for dashboards.
  3. CHT Interoperability Exchanges records with systems like OpenMRS, using FHIR through OpenHIM.

Why this software matters. The CHT supports a million home visits a month, and as of 2024 six governments had selected it for their national community health systems. It's a verified digital public good, licensed under AGPL-3.0 and stewarded by Medic. Hundreds of people contribute to it. Those numbers belong to that community, and the work below is my part of it.

01 · medic/cht-core

CHT Core

Making a large CouchDB deployment cheaper to run, without changing what health workers see.

26 merged PRs on master, Feb 2025 – Nov 2026 · CouchDB, Nouveau, Node.js, Helm

Core is an offline-first framework. Each user's device holds its own copy of their data and replicates it with a CouchDB server, so care continues when the network doesn't. On large deployments, the server's map/reduce indexes become a major cost, because every document written has to pass through every view.

So I looked at the largest views on a sample of instances and asked what each one was really for. Several were expensive to keep and barely used on the server.

Views removed or reduced
ViewWhy it was expensiveWhat replaced itPR
doc_summaries_by_id The largest view sampled: large values for every report and contact. Summaries calculated on demand, as offline users already did. #10831
doc_by_type Emitted every document, but was only queried for rare types. The most common document types are no longer indexed. #10822
docs_by_id_lineage Emitted once for every ancestor of every document with a contact. Two _all_docs calls: the child, then its ancestors. #11319
tasks_by_contact A large index of tasks that only the offline rules engine used. Removed from the server. #11231

Also under load

  • Purging was exhausting the search heap. One purge query could ask Nouveau for 200,000 hits per shard, or up to 2.4 million at once on a 12-shard database. It now makes a single request capped at 20,000. #11494
  • Search was a bottleneck on the sync path. In clustered Kubernetes deployments, all Nouveau traffic went to the first CouchDB node. Each node now talks to its own instance. #11126
  • Purged-document lookups slowed down as purge databases grew. I replaced a filtered _changes request with _all_docs by key. #10396
  • Moving code off direct view queries. I added document-returning getPage and getAll methods, and a phone number qualifier, to cht-datasource. #11214, #11249

02 · medic/cht-sync

CHT Sync

A continuous pipeline from CouchDB to PostgreSQL, so program teams can query what's happening in the field.

22 commits on main, Mar 2024 – Aug 2025 · Node.js, Python, dbt, PostgreSQL, Helm

CouchDB suits replication but not analytics. CHT Sync splits the problem in two: couch2pg follows each database's changes feed and upserts raw documents into PostgreSQL as JSONB, then dbt continuously transforms them into models. The models live in a separate repo, cht-pipeline, so each can change independently.

I built much of the dbt side, including the long-running loop at the centre of it. Any model whose definition changed gets a full refresh, then incremental runs pick up new data. The manifest is stored in PostgreSQL, so changes are still detected after a restart. 

# dbt/dbt-run.py (comments added here)
while True:
    update_models()           # full refresh anything that changed
    run_incremental_models()  # then catch up on new data
    time.sleep(int(os.getenv("DATAEMON_INTERVAL") or 5))

What I built

  • Version 2, for large first syncs. Initial syncs could overwhelm dbt with millions of rows. I added batching, plus selectors so separate services can each run part of the model set. A source column lets several databases share one landing table. #202, #207
  • Change detection. Full refresh for changed models, incremental for everything else, and automatic pickup of pipeline updates. 0869ee9, #130
  • Production deployment. The Helm chart for Kubernetes, with configurable resource limits and dbt thread count. #90, #195
  • Monitoring. An optional SQL exporter, plus queries for sync progress and dbt status. #128

03 · medic/cht-interoperability

CHT Interoperability

Getting a community health worker's records and a clinic's records to agree, through open standards.

51 commits on the openmrs-mediator branch, 2024 – 2025 · TypeScript, HL7 FHIR, OpenHIM, HAPI FHIR

CHT documents reflect each deployment's configuration, so they don't map directly onto FHIR. This project, a team effort built on the OpenHIE architecture, puts a mediator in between to translate and route them through OpenHIM. Its first workflow was loss to follow-up, where an external system asks the CHT to find a patient who has missed care.

My part was extending it to OpenMRS, so that patients and encounters could move both ways between community and facility. On each sync, the mediator compares recently updated resources on the two servers and sorts them by where they exist:

Only in OpenMRS

incoming

Saved to the FHIR server, then on to the CHT.

Only in FHIR

outgoing

Created in OpenMRS.

In both

toupdate

Already on both sides, so no duplicate is created.

The hard parts

  • Infinite loops. In a two-way sync, a record you just received looks like a new change to send back. Each resource is now tagged with its source system.
  • Half-finished encounters. An encounter still in progress could sync with only part of its data, so one with no end time isn't sent until it's finished.
  • Optional by design. Not every project uses OpenMRS, so I split it into its own mediator, with polling set up as an OpenHIM channel. #139

This work lives on a public branch and isn't merged to main. Setup is covered in the CHT docs.