In the absence of a partition, one can ask what else (E) happens and choose between optimizing for latency (L) or consistency (C). Partition tolerance in the CAP sense is always required (because one can't guarantee that a partition won't happen), so the only real choice in CAP is whether to prioritize CP or AP.īut CAP is sometimes extended with other considerations. In Kafka, a partition is simply a division of data to permit parallelism. In CAP, a partition is a rupture in the network that causes two communicating peers to be temporarily unable to communicate. Note: The word partition in CAP is totally different from its use in Kafka streaming. The CAP theorem, which balances consistency (C), availability (A), and partition tolerance (P), is one of the foundational theorems in distributed systems. For each axis, performance lies somewhere between the two ends.įigure 1: Kafka performance involves two orthogonal axes: Availability versus durability and latency versus throughput. The diagram consists of two axes, each with one of the goals at one of its ends. Thus, the main performance considerations for Kafka can be represented as in Figure 1. There are tradeoffs between these two goals as well. Durability (which in our context also includes consistency) determines how robust the entire system is in the face of failure, whereas availability determines how likely it is to be running. Two other opposed goals are durability and availability.
Architectural choices that improve one goal tend to degrade the other, so they can be seen as opposite ends of a spectrum. Latency, in this article, refers to how long it takes to process a single message, whereas throughput refers to how many messages can be processed during a typical time period. Like other systems that handle large amounts of data, Kafka balances the opposing goals of latency and throughput.
#GOLDENRATIO JAVA CODE HOW TO#
This article introduces a way of thinking about the tradeoffs and how to design your message flows using a model I call the Kafka optimization theorem. Similarly, some applications put a premium on durability, whereas others care more about availability. Some prioritize latency over throughput, and some do the opposite. But different applications have different requirements. The performance of your Apache Kafka environment will be affected by many factors, including choices, such as the number of partitions, number of replicas, producer acknowledgments, and message batch sizes you provision.
0 kommentar(er)
