High level architecture

High level architecture#

High level architecture

Fig. 1 High level architecture#

The platform is organized in three tiers: a storage layer, a cache layer, and a web layer.

Storage layer (Kvrocks)#

Kvrocks is the primary persistent storage for all vulnerability data. It runs on TCP (default port 10002) and uses RocksDB as its storage engine, providing a Redis-compatible protocol with significantly lower memory requirements.

Key storage patterns:

  • Vulnerabilities are stored by lowercase ID: cve-2024-1234 → JSON blob

  • Metadata: cve-2024-1234:meta → hash mapping meta types to UUIDs

  • Cross-references: cve-2024-1234:link → set of related vulnerability IDs

  • Source indexes: index:<source_name> → sorted set of vulnerability IDs

  • Last update tracking: last_updates hash with per-source timestamps

Cache layer (Valkey)#

Valkey serves as a fast in-memory cache running on a Unix socket (cache/cache.sock). It handles Pub/Sub messaging between feeders and the web layer, temporary data caching, and real-time event distribution to services such as Meilisearch and the email notification system.

Backend (Feeders)#

The backend consists of 40+ feeders responsible for downloading security advisories from various sources. All feeders inherit from the AbstractFeeder base class (CSAF-based feeders use CSAFGeneric). Each feeder publishes collected data to Valkey Pub/Sub channels, enabling seamless integration with downstream services.

PostgreSQL database#

PostgreSQL stores user accounts, comments, bundles, sightings, watchlists, organizations, and KEV catalogs. The web application uses SQLAlchemy as its ORM with Alembic for schema migrations.

Database models diagram

Fig. 2 Database models diagram#

Web layer (Flask)#

The web interface is a Flask application served by Gunicorn (default port 10001). Views are organized as Blueprints and the REST API is available under /api/. User authentication is handled by Flask-Login with optional two-factor authentication.

When user accounts are enabled (user_accounts: true in config/generic.json), the platform supports comments, bundles, sightings, watchlists, and the CVD workflow.

Tools#

It is possible to extend Vulnerability-Lookup functionalities thanks to its API and various tools. For example:

  • PyVulnerabilityLookup — Python client library for the REST API.

  • FediVuln — publish vulnerability updates to the Fediverse via the Pub/Sub streaming service.

  • ML-Gateway — AI-powered severity classification (VLAI).

Contents