Protocol buffers serialization

What is RPC?

Remote procedure call (English: Remote, Procedure, Call, abbreviated as RPC) is a computer communication protocol. The protocol allows programs running on a computer to call subroutines of another computer, while programmers do not need to program this interaction interactively.
  • 1, in order to allow different clients to access the server, many standardized RPC systems came into being. Most of them use IDL (Interface Description Language) to facilitate cross platform remote procedure calls.
  • 2, the bottom pass data through the socket
  • 3, serialization and serialization are also called encoding and decoding.
  • 4, RPC remote procedure calls, and many RPC frameworks are inter language.
  • 5, RPC calls between networks (serving between services)

RPC development process

  • 1, define an interface specification file that describes a series of information about objects (structures), objects, members, interfaces, methods, and so on.
  • 2, compile the interface specification file into a specific language file through the compiler provided by the RPC framework.
  • 3, you can invoke the remote method just as you call the local method by introducing the files generated by the RPC compiler on both the client and server side.

The efficiency of the RPC framework

The compression ratio of a codec; the speed at which the network is transmitted

Protocol buffers

Protocol buffers is a flexible, efficient, and automated mechanism for serializing structured data. Can be compiled into a variety of source code, such as Java, PHP, python, nodejs, ruby…

  • 1, the proto file is also called idl.
  • 2, the field name does not need to be named after the hump

Sample

Syntax = "proto2"; package com.lihao.netty.nettyprotobuf option; optimize_for = SPEED; option = java_package; java_outer_classname = "com.lihao.netty.nettyprotobuf" option "MyDataInfo"; message Person {required string name =1 required int32 age; optional string = 2; address = 3;}
  • 1, syntax stands for syntax, such as proto2 proto3
  • 2, package represents the package name
  • 3, optimize_for, SPEED, CODE_SIZE, or, and LITE_RUNTIME represent the generators that c++ Java generates code.
  • 4, java_package specifies the package name of the Java. If java_package is set, package does not work, but it is still set.
  • 5, java_outer_classname represents the generated class name
  • 6, message represents the message body
  • 7, the required field description indicates that this parameter must be available
  • 8, the optional field description represents optional.
  • 9, the repeated field describes the repetition of the field, which is the meaning of the set, such as list

Protoc installation and Java introduces java package

1, download the compiler protoc protoc-3.4.0-osx-x86_64.zip
https://github.com/google/protobuf/releases

2, setting environment variables

VI ~/.bash_profile export PATH=$PATH:/Users/lixueqin/common/protoc-3.4.0/bin

3, gradle introduces files

Compile'com.google.protobuf:protobuf-java:3.4.0'compile'com.google.protobuf:protobuf-java-util:3.4.0'

view help

Protoc -h

The.Proto file is stored in the source directory, that is, the Java directory

Srudent.proto

Syntax = "proto2"; package com.lihao.netty.protobuf option; optimize_for = SPEED; option = java_package; java_outer_classname = "com.lihao.netty.protobuf" option "DataInfo"; message Student {required string name =1 required int32 age; optional string = 2; address = 3;}

Generate the corresponding Java class through Srudent.proto

Protoc --java_out=src/main/java src/protobuf/Student.proto

The generated DataInfo.java does not modify it, but it’s good to think of him as a read-only file

Protocol buffers serialization test

Public class ProtoBufTest static void main {public (String... ARG) throws Exception DataInfo.Student (DataInfo.Student.newBuilder) {student =.SetName ("Zhang San").SetAge (28).SetAddress ("Beijing") (.Build); System.out.println (student); / / convert byte can be transmitted on the network byte[] stdent2ByteArray = student.toByteArray (); / / convert the Java object DataInfo.Student student2 = DataInfo.Student.parseFrom (stdent2ByteArray); System.out.println (student2);}}

Netty support for protocol buffers

Example 1:

Patients.proto

Syntax = "proto3"; package com.lihao.netty.api option; optimize_for = SPEED; option = java_package; java_outer_classname = "com.lihao.netty.api" option "Patients"; message Patient {string ID string = 1; name = 2; int32 age = 3;} / / message PatientListResponse repeated Patient returns a list of {patientList} message {PatientListRequest = 1; string uid = 1;} message {PatientDetailRequest string uid = 1; string PID = 2;} message {PatientDetailResponse Patient patient = 1;} message {Api enum ApiType {PatientListResponseType = 0; PatientListRequestType = 1; PatientDetailRequestType = 2; PatientDetailResponseType = 3;} ApiType api_type = 1; / / The same time can access a oneof data_type PatientListResponse PatientListRequest {patientListResponse = 2; patientListRequest = 3; PatientDetailRequest patientDetailRequest PatientDetailResponse = 4; patientDetailResponse = 5;}}

Generate the corresponding java file

Protoc --java_out=src/main/java src/protobuf/Patients.proto
Server
Public class ServerBuf static void main {public (String... ARG) throws Exception bossGroup new NioEventLoopGroup {NioEventLoopGroup = NioEventLoopGroup; workGroup = new) (NioEventLoopGroup) (serverBootstrap = new; try {ServerBootstrap (ServerBootstrap); serverBootstrap.group (bossGroup, workGroup).Handler (New LoggingHandler (LogLevel.INFO).Channel (NioServerSocketChannel.class).ChildHandler (New BufInitialzer) ()); ChannelFuture channelFuture = serverBootstrap.bind (8888) (.Sync); channelFuture.channel (.CloseFuture) (.Sync) (finally);} {(bossGroup.shutdownGracefully); (workGroup.shutdownGracefully);} }
Initializer
Public class BufInitialzer extends ChannelInitializer< SocketChannel> protected void initChannel {@Override (SocketChannel CH) throws Exception (pipeline) {ChannelPipeline = ch.pipeline; pipeline.addLast (New) (ProtobufVarint32LengthFieldPrepender); pipeline.addLast (new (ProtobufVarint32FrameDecoder) pipeline.addLast (New); ProtobufDecoder ((Users.Api.getDefaultInstance)); pipeline.addLast (New) (ProtobufEncoder) (pipeline.addLast); new (BufHandler));}}
Handler
Public class BufHandler extends SimpleChannelInboundHandler< Users.Api> protected void channelRead0 {@Override (ChannelHandlerContext CTX, Users.Api MSG Exception System.out.println (throws) {"server received:"); System.out.println (MSG); if ((msg.hasLoginRequest)) {/ / System.out.println ("landing, landing request request."); Users.Api.Builder (apiBuild = Users.Api.newBuilder) Users.LoginResult; loginResult = Users.LoginResult.newBuilder (.SetUid) ("10001").SetUsername ("Xiaowang").SetAge (18).SetAvatar ("www.leyueq00.com").SetToken ("ewr=234sdf") (.Build); apiBuild.setLoginResult (loginResult); Users.Api back = apiBuil D.build (); ctx.channel ().WriteAndFlush (back);}}

Client code

Client
Public class ClientBuf static void main {public (String... ARG) throws Exception eventLoopGroup new NioEventLoopGroup {NioEventLoopGroup = (try; bootstrap) {Bootstrap = new (Bootstrap); bootstrap.group (eventLoopGroup).Channel (NioSocketChannel.class).Handler (New) (ClientInitializer); ChannelFuture channelFuture = bootstrap.connect (localhost,.Sync) (8888) channelFuture.channel (.CloseFuture); (.Sync) (finally);} {}}} (eventLoopGroup.shutdownGracefully);
Initializer
Public class ClientInitializer extends ChannelInitializer< SocketChannel> protected void initChannel {@Override (SocketChannel CH) throws Exception (pipeline) {ChannelPipeline = ch.pipeline; pipeline.addLast (New) (ProtobufVarint32LengthFieldPrepender); pipeline.addLast (new (ProtobufVarint32FrameDecoder) pipeline.addLast (New); ProtobufDecoder ((Users.Api.getDefaultInstance)); pipeline.addLast (New) (ProtobufEncoder) (pipeline.addLast); new (ClientHandler));}}
Handler
Public class ClientHandler extends SimpleChannelInboundHandler< Users.Api> protected void channelRead0 {@Override (ChannelHandlerContext CTX, Users.Api MSG Exception System.out.println (throws) {"client received:"); System.out.println (MSG) @Override public void;} channelActive (ChannelHandlerContext CTX) throws Exception (apiBuilder) {Users.Api.Builder = Users.Api.newBuilder; Users.LoginRequest = Users.LoginRequest.newBuilder (.SetUsername (loginRequest) the "Lihao").SetPassword ("123") (.Build); apiBuilder.setLoginRequest (loginRequest); Users.Api (API = apiBuilder.build); ctx.channel (.WriteAndFlush) (API);}}

Example two uses oneOf:
Patients.proto

Syntax = "proto3"; package com.lihao.netty.api option; optimize_for = SPEED; option = java_package; java_outer_classname = "com.lihao.netty.api" option "Patients"; message Patient {string ID string = 1; name = 2; int32 age = 3;} / / message PatientListResponse repeated Patient returns a list of {patientList} message {PatientListRequest = 1; string uid = 1;} message {PatientDetailRequest string uid = 1; string PID = 2;} message {PatientDetailResponse Patient patient = 1;} message {Api enum ApiType {PatientListResponseType = 0; PatientListRequestType = 1; PatientDetailRequestType = 2; PatientDetailResponseType = 3;} ApiType api_type = 1; / / The same time can access a oneof data_type PatientListResponse PatientListRequest {patientListResponse = 2; patientListRequest = 3; PatientDetailRequest patientDetailRequest PatientDetailResponse = 4; patientDetailResponse = 5;}}
Server handler
Public class BufHandlerOneof extends SimpleChannelInboundHandler< Patients.Api> protected void channelRead0 {@Override (ChannelHandlerContext CTX, Patients.Api MSG Exception System.out.println (throws) {"server received: +msg); Channel channel = ctx.channel (switch); (msg.getApiType) (case PatientListRequestType:) {Patients.PatientListResponse.Builder = Patients.PatientListResponse.newBuilder (listresponseBuild); listresponseBuild.addPatientList (Patients.Patient.newBuilder (.SetId) (" 10001 ").SetName (" Wang Xi Ya ").SetAge (20)); listresponseBuild.addPatientList (Patients.Patient.newBuilder (.SetId) (" 10002 ").SetName (" Xiao Wang ").SetAge (24)); ListresponseBuild.addPatientList (Patients.Patient.newBuilder (.SetId) ("10003").SetName ("Li Zhi Min").SetAge (29)); Patients.Api.Builder builder = Patients.Api.newBuilder (.SetApiType) (Patients.Api.ApiType.PatientListResponseType) (.SetPatientListResponse) (listresponseBuild.build); channel.writeAndFlush (builder.build) (break); case PatientDetailRequestType:; Patients.PatientDetailResponse.Builder detailBuild = Patients.PatientDetailResponse.newBuilder (.SetPatient) (Patients.Patient.newBuilder ().SetId ("10001").SetName ("Wang Xi Ya").SetAge (20)); builder = Patients.Api.newBuilder (.SetApiType (Patients.Api.ApiType.Pat) IentDetailResponseType).SetPatientDetailResponse (detailBuild); channel.writeAndFlush (builder.build ()); break;}}
Client handler
Public class ClientHandlerOneof extends SimpleChannelInboundHandler< Patients.Api> protected void channelRead0 {@Override (ChannelHandlerContext CTX, Patients.Api MSG Exception System.out.println (throws) {"client received:"); System.out.println (MSG) @Override public void;} channelActive (ChannelHandlerContext CTX) throws Exception random new Random {Random = (for (int); I = 0; I < 5; i++) {int value = random.nextInt (< = 3); 1? 1: 2; System.out.println ("random" + value); if (value = = 1) {Patients.Api requestlist = Patients.Api.newBuilder (.SetApiType) (Patients.Api.ApiType.PatientListRequestType) .setPatientListRequest (Patients.PatientListRequest.newBuilder).SetUid ((1001)) (.Build); ctx.channel (.WriteAndFlush) else (requestlist);} if (value = = 2) {Patients.Api = Patients.Api.newBuilder (detailReuset).SetApiType (Patients.Api.ApiType.PatientDetailRequestType).SetPatientDetailRequest (Patients.PatientDetailRequest.newBuilder) (.SetPid (2001).SetUid (1001)) (.Build) ctx.channel (.WriteAndFlush); (detailReuset);} else {Patients.Api = Patients.Api.newBuilder (requestlist).SetApiType (Patients.Api.ApiType.PatientListRequestType) (.SetPatientListRequest (Patients.PatientListRequest.newBuilder). SetUid (10012)) (.Build); ctx.channel (.WriteAndFlush) (requestlist);} System.out.println ("C");}}}

Reference document

Https://github.com/google/protobuf/tree/master/java
https://developers.google.com/protocol-buffers/docs/proto