Microsoft Fabric offers several ways to process streaming data, and the right engine depends on latency, transformation complexity, state, developer skill, and the destination. For independent DP-700 architecture practice, the important options include Eventstream for managed ingestion and routing, Spark Structured Streaming for code-first stateful processing, and KQL/Eventhouse for real-time analytics over high-volume event data.

These tools overlap but are not identical. A low-code event-routing flow does not need the same engine as a multi-stream stateful join, and a real-time operations dashboard has different needs from a durable Delta pipeline. The decision should begin with the event lifecycle: where events originate, how much transformation is required, how late data is handled, where state lives, and what downstream consumers need.

Eventstream is the managed ingestion and routing layer

Fabric Eventstream provides a managed visual experience for connecting streaming sources, applying selected event-processing transformations, and routing data to destinations such as Eventhouse, lakehouse, custom endpoints, and Spark Notebook destinations that Microsoft currently designates as preview. Source ownership still matters: document who controls the event contract, authentication, partitioning, retention, and schema evolution so an upstream producer cannot break multiple destinations with an undocumented change. It is a strong fit when teams need low-code ingestion and routing without writing a long-running Spark application. Within Microsoft Fabric Real-Time Intelligence, Eventstreams provide a managed path for ingesting, transforming, and routing events before downstream KQL or other consumers process them.

Use Eventstream when the event path is understandable as sources, transformations, and destinations. Eventstream can perform filtering, field management, aggregation, grouping, and other supported event-processing operations before routing. Keep these transformations focused on stream shaping and routing; when logic becomes state-heavy, highly customized, or dependent on external libraries, Spark is usually easier to test and maintain. It can simplify connector management and centralize real-time flows, but very complex custom state or specialized libraries may still justify a code-first processor.

KQL and Eventhouse are optimized for real-time analytics

Eventhouse stores high-volume event and telemetry data for low-latency analytical querying with Kusto Query Language. Use ingestion mappings and table policies consistently so event fields remain queryable as schemas evolve. For semi-structured payloads, preserve enough original content for investigation while extracting the fields analysts use frequently. KQL is designed for filtering, parsing, aggregation, time-window analysis, anomaly investigation, and operational exploration over event-oriented data.

This is a strong fit for observability, security telemetry, IoT, logs, and other scenarios where analysts need to query recent and historical events interactively. Eventhouse can support dashboards and operational investigations where analysts repeatedly slice by time, device, service, user, or event type. Retention, ingestion mapping, schema, and table design should be treated as part of the real-time architecture rather than afterthoughts. The workload is analytical rather than transactional: Eventhouse is optimized for fast event analysis, not for replacing an OLTP database.

Spark Structured Streaming is the code-first stateful engine

Spark Structured Streaming treats an incoming stream as an unbounded table and lets engineers use DataFrame and Dataset-style APIs to transform new data incrementally. Keep source offsets, checkpoint location, output table, and application code version tied together operationally. Reusing a checkpoint with materially different query logic can produce state incompatibilities or misleading recovery assumptions. In Fabric it can read from supported streaming systems, files, Delta tables, Event Hubs, Kafka-compatible endpoints, and Fabric Eventstream sources.

Choose Structured Streaming when the workload needs programmatic transformations, stateful aggregations, custom libraries, stream-stream joins, complex enrichment, or durable Delta output. Fabric also supports scheduled or available-now patterns for Structured Streaming workloads that should process all currently available data and then stop, which is useful when a continuously running job is unnecessary. It brings more flexibility than low-code Eventstream at the cost of code ownership, checkpoint management, and Spark operations.

Micro-batch and real-time execution serve different latency goals

Structured Streaming commonly runs in micro-batch mode, grouping incoming records into small batches and committing results repeatedly. This provides strong general-purpose performance and integrates well with Delta sinks. Shorter intervals reduce latency but can create more commits and smaller files.

Fabric also supports a lower-latency Real-time Mode for supported Structured Streaming workloads on Fabric Runtime 2.0 (Spark 4.1) or later; it is not a drop-in option for every source, transformation, or sink. Use it only when the business actually needs record-level low latency. Micro-batch remains the default starting point because it usually offers a simpler throughput, state, and cost balance. Tune trigger intervals from measured arrival rate and processing duration. If each micro-batch consistently takes longer than the configured interval, latency will grow even though the stream remains technically healthy. Very small intervals can also increase commit overhead and create many small output files.

Stateful operations need watermarks and checkpoints

Aggregations, deduplication, stream-stream joins, and custom state require the engine to remember information across processing intervals. Structured Streaming uses checkpoints to preserve source progress and state, while watermarks help bound state when late data is possible.

Choose event-time policy deliberately. Watermarks should be attached to the event-time column that represents the business event, not merely ingestion time, when state eviction depends on event lateness. For deduplication and stream-stream joins, validate what happens to records that arrive after the watermark because those records may be discarded rather than corrected later. A watermark that is too aggressive drops legitimate late events; one that is too generous allows state to grow and increases resource use. Measure real arrival delay and define lateness according to the business requirement rather than copying a default value.

Durable Delta output is a strong handoff point

Streaming workloads often become easier to operate when raw or lightly processed events are written to Delta tables before many downstream consumers branch off. Preserve event time, ingestion time, source identifier, and any unique event key so downstream teams can measure lateness, deduplicate replay, and trace a record back to the originating system. A durable bronze stream provides replay, separates failure domains, and lets downstream jobs run at different schedules.

This pattern adds some storage and latency, but it avoids forcing every consumer to read the original event system independently. Use checkpointed, idempotent writes so restart does not duplicate output. Partition and compact Delta data according to downstream access patterns; a streaming job that writes tiny files every few seconds can shift the performance problem from ingestion to later analytics. It also gives batch and streaming workloads a shared OneLake representation that can be queried by Spark, SQL, semantic models, and other Fabric experiences.

KQL, Eventstream, and Spark can be combined

A common architecture uses Eventstream to connect sources and route events, Eventhouse/KQL for operational real-time analytics, and Spark Structured Streaming for custom stateful processing or curated Delta output. Another valid architecture may use only Eventstream and Eventhouse, or only Structured Streaming into Delta. The smallest architecture that satisfies latency, transformation, replay, and query requirements is usually easier to operate. These are complementary layers when each has a clear responsibility.

Do not create a chain merely because all tools are available. Every extra hop adds latency, monitoring, permissions, and failure states. Keep the path as short as possible while still meeting transformation, replay, analytics, and governance requirements.

Production streaming needs explicit fault tolerance

Structured Streaming checkpoints should be durable and unique per query. Checkpoint compatibility can become important after code or schema changes, so deployment runbooks should define whether a query can restart from the existing checkpoint or requires a controlled reset and replay. Never delete a production checkpoint casually just to clear an error; doing so can replay or skip data depending on source and sink behavior. Microsoft advises against sharing one checkpoint across independent streaming queries. Sink logic used through foreachBatch should be idempotent because a microbatch can be retried.

Eventstream and Eventhouse also need health monitoring for source connectivity, ingestion, schema, destination failures, and latency. Include backlog or end-to-end lag in the service objective, not merely process uptime. A stream that is connected but twenty minutes behind may violate the business requirement even though every component reports as running. Streaming is a continuously operating system: a pipeline that was correct at deployment can become stale, backlogged, or broken after a source schema or credential changes. Define ownership for source contracts and test schema evolution before rollout. Add dead-letter or quarantine handling where the architecture supports it so malformed events can be investigated without forcing the whole stream to stop or silently dropping evidence.

Choose the engine from the workload, then measure it in production

The DP-700 Fabric data-engineering scope asks engineers to choose among streaming engines because the best answer changes with the workload. Eventstream favors managed low-code routing, KQL/Eventhouse favors real-time analytical exploration, and Spark Structured Streaming favors programmable stateful processing.

Define target latency, event volume, state requirements, transformation complexity, replay needs, and downstream destinations before choosing the tool. Include team operating skills and failure-recovery expectations. Code-first Spark provides flexibility, while managed Eventstream and Eventhouse can reduce custom operational work. The right engine is the one the team can keep healthy under normal load, late data, schema changes, restart scenarios, and predictable, testable, documented production recovery. Then monitor actual lag, throughput, errors, state growth, checkpoint health, and cost. Streaming architecture is successful when it meets freshness goals predictably, remains recoverable, and does not depend on the most real-time-sounding technology.

Streaming design also needs an event-time policy. Decide how long late events are accepted, where checkpoints and replayable source data are retained, and whether downstream tables can tolerate corrections. In Real-Time Intelligence, Eventstreams can simplify managed ingestion and routing, while KQL or Spark handles different styles of stateful processing. The right combination is the one whose recovery and late-data behavior is explicit under failure, not merely the one with the lowest demo latency.