# Message Broker: Pulsar [EAP](/content/docs/hdp_product/maturity_model.html)

See the [Message Broker](/content/docs/guide_installation/message_broker.html) page for a description of how message queues work by default in Smile CDR.

[Apache Pulsar](https://pulsar.apache.org/) is a distributed messaging and streaming platform designed for high performance, low latency, and high availability. Smile CDR supports Pulsar as a message broker option for subscription processing, batch processing, mdm processing, and other asynchronous tasks that require a message broker.

Set [`module.clustermgr.config.messagebroker.type`](/content/docs/configuration_categories/clustermgr_message_broker.html#property-message-broker-type/index.html) to `PULSAR` in the Cluster Manager to have Smile CDR use Pulsar instead of a JMS message broker or Kafka for subscription processing.

Smile CDR uses the following configuration properties for Pulsar configuration:

```properties
module.clustermgr.config.messagebroker.type=PULSAR
module.clustermgr.config.pulsar.service.url=pulsar://localhost:6650
module.clustermgr.config.pulsar.admin.url=http://localhost:8080
module.clustermgr.config.pulsar.ack_timeout_seconds=30
module.clustermgr.config.pulsar.receiver.queue.size=100
module.clustermgr.config.pulsar.validate_topics_exist_before_use=false
```

Copy

The values above for the `module.clustermgr.config.pulsar.*` properties are the defaults, so you don't need to include these properties in your config if you are using the default value.

`pulsar.service.url` is the URL for the Pulsar service. The default is `pulsar://localhost:6650` for a local Pulsar instance.

`pulsar.admin.url` is the URL for the Pulsar admin service. The default is `http://localhost:8080` for a local Pulsar instance.

`pulsar.ack_timeout_seconds` specifies the timeout in seconds for acknowledging messages. The default is 30 seconds.

`pulsar.receiver.queue.size` specifies the size of the receiver queue. The default is 100.

`pulsar.validate_topics_exist_before_use`. Set this to true if your Pulsar broker is configured to prevent new topics from being automatically created. When this property is set to true, Smile will prevent subscriptions from being created or updated if the delivery topic they depend on doesn't exist yet.

Like the JMS Message Broker and Kafka, Pulsar uses the Cluster Manager module `consumers_per_matching_queue` and `consumers_per_delivery_queue` properties to determine the number of consumers for each topic.

# Pulsar Topic Names

For Pulsar brokers, the Smile CDR channels are synonymous with Pulsar topics. See [Channel Names](/content/docs/guide_installation/message_broker.html#channel-names/index.html) for details on how these topics will be named.

# Message Ordering with Message Keys

Pulsar guarantees message ordering within a partition. To ensure related messages are processed in order, assign the same **message key** to messages that must be processed sequentially.

When a message is published with a message key, Pulsar hashes the key to determine which partition receives the message. All messages with the same key hash to the same partition, ensuring they are consumed in order.

## Message Key Selection

Choose a message key based on the ordering requirements of your data:

- **Patient-centric data**: Use the patient identifier (e.g., MRN, enterprise ID) as the message key
- **Non-patient-centric data**: Use the resource ID or another stable identifier. For low-volume non-patient data, a shared constant key routes all such messages to a single partition.

## Trade-offs: Ordering vs. Parallelism

Message keys affect parallelism:

- **More unique keys** = messages distributed across more partitions = higher parallelism
- **Fewer unique keys** = messages concentrated in fewer partitions = stronger ordering guarantees but less parallelism

For example, using patient ID as the message key allows parallel processing across different patients while ensuring all messages for the same patient are processed in order.

For guidance on setting message keys in Camel routes, see [Message Keys for Ordering](/content/docs/camel/smile_camel_processors_cluster_manager.html#message-keys/index.html).

# Customizing Pulsar Configuration with Interceptors

Smile CDR provides interceptor hooks that allow you to customize Pulsar configuration beyond what is available through the standard configuration properties. These interceptors give you direct access to the Pulsar client's [ConsumerBuilder](https://pulsar.apache.org/api/client/4.0.x/org/apache/pulsar/client/api/ConsumerBuilder.html) and [ProducerBuilder](https://pulsar.apache.org/api/client/4.0.x/org/apache/pulsar/client/api/ProducerBuilder.html) objects before they are used to create consumers and producers.

## Consumer Configuration

The `PULSAR_CONSUMER_BUILDER` interceptor hook allows you to modify the Pulsar ConsumerBuilder before it is used to create a consumer.

Example interceptor implementation:

```java
import ca.cdr.api.fhir.interceptor.CdrHook;
import ca.cdr.api.fhir.interceptor.CdrPointcut;
import ca.uhn.fhir.interceptor.api.Interceptor;
import org.apache.pulsar.client.api.ConsumerBuilder;
import org.apache.pulsar.client.api.ConsumerCryptoFailureAction;

@Interceptor
public class PulsarConsumerCustomizer {

@CdrHook(CdrPointcut.PULSAR_CONSUMER_BUILDER)
    public void customizeConsumerBuilder(ConsumerBuilder<?> theConsumerBuilder) {
        // Customize the consumer builder
        theConsumerBuilder
            .cryptoFailureAction(ConsumerCryptoFailureAction.CONSUME)
            .maxTotalReceiverQueueSizeAcrossPartitions(50000)
            .autoUpdatePartitions(true);
    }
}
```

Copy

## Producer Configuration

The `PULSAR_PRODUCER_BUILDER` interceptor hook allows you to modify the Pulsar ProducerBuilder before it is used to create a producer.

Example interceptor implementation:

```java
import ca.cdr.api.fhir.interceptor.CdrHook;
import ca.cdr.api.fhir.interceptor.CdrPointcut;
import ca.uhn.fhir.interceptor.api.Interceptor;
import org.apache.pulsar.client.api.ProducerBuilder;
import org.apache.pulsar.client.api.CompressionType;

@Interceptor
public class PulsarProducerCustomizer {

@CdrHook(CdrPointcut.PULSAR_PRODUCER_BUILDER)
    public void customizeProducerBuilder(ProducerBuilder<?> theProducerBuilder) {
        // Customize the producer builder
        theProducerBuilder
            .compressionType(CompressionType.ZSTD)
            .blockIfQueueFull(true)
            .enableBatching(true)
            .batchingMaxPublishDelay(10, java.util.concurrent.TimeUnit.MILLISECONDS)
            .batchingMaxMessages(1000);
    }
}
```

Copy

To use these interceptors, you need to register them on the Smile CDR Cluster Manager Module. See the [Interceptors](/content/docs/interceptors/interceptors.html) documentation for more information on how to register interceptors.

# Dead Letter Queue

For Pulsar, the Dead Letter Queue (DLQ) is named `PULSAR.DLQ`. This is where messages that cannot be delivered after the configured number of retries will be sent. See [Dead Letter Queues](/content/docs/subscription/introduction.html#dlq/index.html) for more information.

[Message Broker: ActiveMQ](/content/docs/guide_installation/activemq.html)

Message Broker: Pulsar

Installing

[Platform Requirements](/content/docs/guide_installation/platform_requirements.html)
[Preparing a Linux Host](/content/docs/guide_installation/preparing_a_linux_host.html)
[Installing Smile CDR, NGINX and PostgreSQL in a Docker Stack](/content/docs/guide_installation/installing_smilecdr_nginx_and_postgresql_in_a_docker_stack.html)
[Deploying a Kubernetes Managed Cluster](/content/docs/guide_installation/deploying_a_kubernetes_managed_cluster.html)
[Docker Container Installation](/content/docs/guide_installation/docker_container_installation.html)
[Configuring Smile CDR](/content/docs/guide_installation/configuring_smile_cdr.html)
[Unix Service Installation](/content/docs/guide_installation/unix_service_installation.html)
[Tuning your Installation](/content/docs/guide_installation/tuning_your_installation.html)
[Designing a Cluster](/content/docs/guide_installation/designing_a_cluster.html)
[Message Brokers](/content/docs/guide_installation/message_broker.html)
[Message Broker Failure Management](/content/docs/guide_installation/message_broker_failure_management.html)
[Message Broker: Kafka](/content/docs/guide_installation/kafka.html)
[Message Broker: ActiveMQ](/content/docs/guide_installation/activemq.html)
[Message Broker: Pulsar](/content/docs/guide_installation/pulsar.html)
[Pre-Seeding Configuration and Data](/content/docs/guide_installation/pre_seeding.html)
[Production Checklist](/content/docs/guide_installation/production_checklist.html)
[Module Licensing](/content/docs/guide_installation/licensing.html)

[Pre-Seeding Configuration and Data](/content/docs/guide_installation/pre_seeding.html)