Using Kafka in an Express.js application for tracking flight arrival and departure status

20 min read

What?

Using Kafka in an Express.js application for tracking flight arrival and departure status requires setting up a Kafka broker, creating producers to publish messages (flight status updates in this case) to Kafka, and consumers to consume these messages.

Below is a simplified example of how you can achieve this. Please note that integrating Kafka in a real-world application requires more detailed error handling and optimizations.

1. Set Up Kafka:

If you haven't already set up Kafka, you will need to download and run a Kafka broker. Here's a brief outline of the steps:

Download and extract the Kafka binaries from the official Apache Kafka website.
Start the ZooKeeper server: bin/zookeeper-server-start.sh config/zookeeper.properties
Start the Kafka broker: bin/kafka-server-start.sh config/server.properties

2. Create Topics:

Create a topic for flight arrivals and another for departures:


    bin/kafka-topics.sh --create --topic flight-arrivals --bootstrap-server localhost:9092
    bin/kafka-topics.sh --create --topic flight-departures --bootstrap-server localhost:9092

3. Express.js Application:

a. Setup a new Express.js application:

    
npm init -y
npm install express kafka-node

b. Write the app:


const express = require('express');
const { KafkaClient, Producer, Consumer } = require('kafka-node');

const app = express();
const port = 3000;

const kafkaClient = new KafkaClient({ kafkaHost: 'localhost:9092' });
const kafkaProducer = new Producer(kafkaClient);
const kafkaConsumerArrivals = new Consumer(kafkaClient, [{ topic: 'flight-arrivals' }]);
const kafkaConsumerDepartures = new Consumer(kafkaClient, [{ topic: 'flight-departures' }]);

app.use(express.json());

app.post('/flight/arrival', (req, res) => {
    kafkaProducer.send([{ topic: 'flight-arrivals', messages: JSON.stringify(req.body) }], (err) => {
        if (err) {
            res.status(500).send("Error producing message");
            return;
        }
        res.status(200).send("Flight arrival message sent");
    });
});

app.post('/flight/departure', (req, res) => {
    kafkaProducer.send([{ topic: 'flight-departures', messages: JSON.stringify(req.body) }], (err) => {
        if (err) {
            res.status(500).send("Error producing message");
            return;
        }
        res.status(200).send("Flight departure message sent");
    });
});

kafkaConsumerArrivals.on('message', (message) => {
    console.log('Flight Arrived:', JSON.parse(message.value));
});

kafkaConsumerDepartures.on('message', (message) => {
    console.log('Flight Departed:', JSON.parse(message.value));
});

app.listen(port, () => {
    console.log(`Express server running on http://localhost:${port}`);
});

This Express app allows you to send POST requests with flight status updates to /flight/arrival and /flight/departure. It will then publish these updates to the respective Kafka topics.

A Kafka consumer then listens for messages from the topics and logs them to the console.

Let me tell you

We used kafka-node library to interact with Kafka from our Express.js application.
We defined two endpoints in our app for flight arrivals and departures where users can POST flight data.
Upon receiving the flight data, the app publishes this data to the corresponding Kafka topic.
Consumers listen for messages from these topics. When a message is received, it logs the flight status to the console.

Please note this is a basic setup. In a real-world application, you would probably want more consumers for scalability, better error handling, and other optimizations.

This is me…