Skip to main content


Showing posts with the label Apache AVRO example

Apache Avro - RPC framework

This post is in continuation to my previous posts on Apache Avro - Introduction , Apache Avro - Generating classes from Schema and Apache Avro - Serialization . In this post, we will share insights on using Apache Avro as RPC framework. We first need to define a protocol to use Apache Avro as RPC framework. Before going into depth of this topic, let's discuss What protocol is? Avro protocols describes RPC interfaces. They are defined as JSON similar to Schema . A protocol has following attributes protocol : a string, defining name of the protocol. namespace : an optional that qualifies the name. types : an optional list of definitions of named types (like record, enum, fixed and errors). messages : an optional JSON object whose keys are method names of protocoland whose values are objects whose attributes are described below. No two messages may have the same name. Further, Message have following attributes request : a list of named, typed parameter schemas. respon

Apache Avro - Introduction

In this post, we will discuss following items What is Apache Avro? What is Avro schema and how to define it? Serialization in Apache Avro. What is Apache Avro ? " Apache Avro is data serialization library " That's it, huh. This is what you will see when you open their official page. Apache Avro is: Schema based data serialization library. RPC framework (support). Rich data structures (Primary includes null, string, number, boolean and Complex includes Record, Array, Map etc.). A compact, fast and binary data format. What is Avro schema and how to define it? Apache Avro serialization concept is based on Schema. When you write data, schema is written along with it. When you read data, schema will always be present. The schema along with data makes it fully self describing. Schema is representation of AVRO datum(Record). It is of two types: Primitive and Complex . Primitive types These are the basic type supported by Avro. It includes null, int, long, byte