由于项目需求，准备在嵌入式上使用rpc方案，调研了多个方案，最终由于Grpc和protobuf天然的亲和性，决定对Grpc进行移植。

Grpc的交叉编译支持三种方式：bazel，cmake，makefile；bazel 由于编译太麻烦，主要适合google内部，直接放弃了，本文主要讲解基于cmake的交叉编译方式，makefile 也是类似方式。

一交叉编译

1. 首先下载Grpc：

git clone git@github.com:grpc/grpc.git

2. 由于Grpc 依赖较多的第三方库，我不想在系统中都安装，所以更新submodule

git submodule update --init

3. 交叉编译，grpc 交叉编译比较麻烦，主要原因时grpc 中包含了很多动态生成的操作，比如：使用protoc 生成protobuf的code，使用protoc-plugin 生成rpc的code；这些操作都时在编译时执行的，而我们编译的系统时ARM上的所以运行会报错，解决这些问题的办法就是关闭无用的处理，或替换为X86的工具，具体命令如下：

make build; // 创建编译临时目录

cmake -DCMAKE_TOOLCHAIN_FILE=XXX.cmake -DgRPC_PROTOBUF_PROVIDER=package -DgRPC_PROTOBUF_PACKAGE_TYPE=MODULE -DProtobuf_PROTOC_EXECUTABLE=/usr/local/bin/protoc -DgRPC_BUILD_GRPC_PYTHON_PLUGIN=OFF -DABSL_RUN_TESTS=OFF ../

实际编译时，由于有些工具会依赖protoc，会报很多编译的错误，可以使用以下命令，关闭所有plugin

cmake -DCMAKE_TOOLCHAIN_FILE=XXXX.cmake -DgRPC_PROTOBUF_PROVIDER=package -DgRPC_PROTOBUF_PACKAGE_TYPE=MODULE -DProtobuf_PROTOC_EXECUTABLE=/usr/local/bin/protoc -DgRPC_BUILD_GRPC_PYTHON_PLUGIN=OFF -DgRPC_BUILD_GRPC_PHP_PLUGIN=OFF -DgRPC_BUILD_GRPC_OBJECTIVE_C_PLUGIN=OFF -DgRPC_BUILD_GRPC_NODE_PLUGIN=OFF -DgRPC_BUILD_GRPC_CSHARP_PLUGIN=OFF -DgRPC_BUILD_GRPC_CPP_PLUGIN=OFF -DgRPC_BUILD_CODEGEN=OFF -DgRPC_BUILD_CSHARP_EXT=OFF -DgRPC_BUILD_GRPC_RUBY_PLUGIN=OFF -DABSL_RUN_TESTS=OFF -DBUILD_TESTING=OFF -DCARES_BUILD_TOOLS=OFF -DCMAKE_INSTALL_PREFIX=XXXX ..

此外，还有几个工具编译也需要屏蔽以下，CMAKE没有选项只能屏蔽CMakeLists

PS：此处将protobuf 设置为X86安装的protoc工具，其中package 表示直接从系统中查找，否则要编译grpc自带的third，需要设置为module，参照：https://stackoverflow.com/questions/52202453/cross-compiling-grpc-using-cmake

4. 执行编译操作，我们只需要使用rpc功能，所以只编译一个模块

cmake --build . --target grpc++_unsecure

因为是内部通信，所以我编译的时unsecure，否则可以编译

cmake --build . --target grpc++

PS：此处要说明一下，如果想查看有哪些可以编译的模块，可以执行 make help

5. 目前编译 grcp++_reflection 会报错：_gRPC_CPP_PLUGIN-NOTFOUND: program not found or is not executable

由于我没用到就放弃了

二执行测试用例

1.编译protobuf库

cmake -DCMAKE_TOOLCHAIN_FILE=XXX.cmake -Dprotobuf_BUILD_TESTS=OFF ..

2. 编写protobuf 文件

包含rpc的protobuf 文件和常规的唯一区别，就时定义一个rpc 的service，用例如下：

syntax = "proto3";

option java_multiple_files = true;
option java_package = "io.grpc.examples.helloworld";
option java_outer_classname = "HelloWorldProto";
option objc_class_prefix = "HLW";

package helloworld;

// The greeting service definition.
service Greeter {
  // Sends a greeting
  rpc SayHello (HelloRequest) returns (HelloReply) {}
}

// The request message containing the user's name.
message HelloRequest {
  string name = 1;
}

// The response message containing the greetings
message HelloReply {
  string message = 1;
}

3. 生成测试用例

grpc 提供了hello world的测试用例，在使用时，需要注意一下，protobuf 生成rpc code 分为两步：

（1）生成protobuf 对应的code

protoc --cpp_out=proto_files/ --proto_path=proto_files/ --proto_pth=helloworld/ helloworld/helloworld.proto

（2）生成rpc 对应的code

protoc --plugin=protoc-gen-grpc-cpp=../../grpc_cpp_plugin --grpc-cpp_out=proto_files/ --proto_pth=helloworld/ helloworld/helloworld.proto

3. 编译测试用例

由于原生的CMake需要依赖本地编译的grpc，也就是CMAKE_INSTALL_PREFIX，所以我就简单写了个CMakeLists，可以看到有很多依赖库，为了深入静态库顺序的麻烦，我使用了-Wl,--start-group 和 -Wl,--end-group的选项，具体代码如下：

cmake_minimum_required(VERSION 3.5.1)

project(helloworld)

set(CMAKE_CXX_STANDARD 14)

include_directories(./  ./include/)
link_directories(./ ./lib)

add_executable(greeter_server greeter_server.cc helloworld.grpc.pb.cc helloworld.pb.cc)
target_link_libraries(greeter_server PRIVATE  
    -Wl,--start-group
    grpc++_unsecure
    grpc_unsecure
    gpr upb 
    protobuf 
    protobuf-lite
    re2 cares 
    absl_dynamic_annotations
    absl_log_severity
    absl_raw_logging_internal
    absl_spinlock_wait
    absl_throw_delegate
    absl_int128
    absl_strings
    absl_strings_internal
    absl_str_format_internal
    absl_civil_time
    absl_time 
    absl_time_zone 
    absl_bad_optional_access 
    address_sorting
    -Wl,--end-group
    z pthread dl)

add_executable(greeter_client greeter_client.cc helloworld.grpc.pb.cc helloworld.pb.cc)
target_link_libraries(greeter_client PRIVATE
    -Wl,--start-group
    grpc++_unsecure
    grpc_unsecure
    gpr upb 
    protobuf 
    protobuf-lite
    re2 cares 
    absl_dynamic_annotations
    absl_log_severity
    absl_raw_logging_internal
    absl_spinlock_wait
    absl_throw_delegate
    absl_int128
    absl_strings
    absl_strings_internal
    absl_str_format_internal
    absl_civil_time
    absl_time 
    absl_time_zone 
    absl_bad_optional_access 
    address_sorting
    -Wl,--end-group
    z pthread dl)

cmake -DCMAKE_TOOLCHAIN_FILE=XXX.cmake ..

三执行

执行 ./greeter_client, 以下输出表示成功。

------------- begin=1595339577-961890 ------------ !!!
------------- end=1595339577-962887 ------------ !!!

Greeter received: Hello world

我们可以看到通信的耗时，还是很长的，基本时ms级别的

四性能profile

GRpc 提供了profile的功能，但是并没有显示的开关操作，我通过修改代码打开的，运行后会存储latency_trace.txt 的文件；

使用GRpc提供的python脚本可以格式显示，脚本位置为：grpc/tools/profiling/

由于GRpc 提供的时Python2的脚本，为了适配Python3我进行了部分修改，修改后的脚本如下：

#!/usr/bin/env python2.7
# Copyright 2015 gRPC authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import argparse
import collections
import hashlib
import itertools
import json
import math
import sys
import tabulate
import time

SELF_TIME = object()
TIME_FROM_SCOPE_START = object()
TIME_TO_SCOPE_END = object()
TIME_FROM_STACK_START = object()
TIME_TO_STACK_END = object()
TIME_FROM_LAST_IMPORTANT = object()

argp = argparse.ArgumentParser(
    description='Process output of basic_prof builds')
argp.add_argument('--source', default='latency_trace.txt', type=str)
argp.add_argument('--fmt', choices=tabulate.tabulate_formats, default='simple')
argp.add_argument('--out', default='-', type=str)
args = argp.parse_args()


class LineItem(object):

    def __init__(self, line, indent):
        self.tag = line['tag']
        self.indent = indent
        self.start_time = line['t']
        self.end_time = None
        self.important = line['imp']
        self.filename = line['file']
        self.fileline = line['line']
        self.times = {}


class ScopeBuilder(object):

    def __init__(self, call_stack_builder, line):
        self.call_stack_builder = call_stack_builder
        self.indent = len(call_stack_builder.stk)
        self.top_line = LineItem(line, self.indent)
        call_stack_builder.lines.append(self.top_line)
        self.first_child_pos = len(call_stack_builder.lines)

    def mark(self, line):
        line_item = LineItem(line, self.indent + 1)
        line_item.end_time = line_item.start_time
        self.call_stack_builder.lines.append(line_item)

    def finish(self, line):
        assert line['tag'] == self.top_line.tag, (
            'expected %s, got %s; thread=%s; t0=%f t1=%f' %
            (self.top_line.tag, line['tag'], line['thd'],
             self.top_line.start_time, line['t']))
        final_time_stamp = line['t']
        assert self.top_line.end_time is None
        self.top_line.end_time = final_time_stamp
        self.top_line.important = self.top_line.important or line['imp']
        assert SELF_TIME not in self.top_line.times
        self.top_line.times[
            SELF_TIME] = final_time_stamp - self.top_line.start_time
        for line in self.call_stack_builder.lines[self.first_child_pos:]:
            if TIME_FROM_SCOPE_START not in line.times:
                line.times[
                    TIME_FROM_SCOPE_START] = line.start_time - self.top_line.start_time
                line.times[TIME_TO_SCOPE_END] = final_time_stamp - line.end_time


class CallStackBuilder(object):

    def __init__(self):
        self.stk = []
        self.signature = hashlib.md5()
        self.lines = []

    def finish(self):
        start_time = self.lines[0].start_time
        end_time = self.lines[0].end_time
        self.signature = self.signature.hexdigest()
        last_important = start_time
        for line in self.lines:
            line.times[TIME_FROM_STACK_START] = line.start_time - start_time
            line.times[TIME_TO_STACK_END] = end_time - line.end_time
            line.times[
                TIME_FROM_LAST_IMPORTANT] = line.start_time - last_important
            if line.important:
                last_important = line.end_time
        last_important = end_time

    def add(self, line):
        line_type = line['type']
        self.signature.update(line_type.encode('utf-8'))
        self.signature.update(line['tag'].encode('utf-8'))
        if line_type == '{':
            self.stk.append(ScopeBuilder(self, line))
            return False
        elif line_type == '}':
            assert self.stk, (
                'expected non-empty stack for closing %s; thread=%s; t=%f' %
                (line['tag'], line['thd'], line['t']))
            self.stk.pop().finish(line)
            if not self.stk:
                self.finish()
                return True
            return False
        elif line_type == '.' or line_type == '!':
            if self.stk:
                self.stk[-1].mark(line)
            return False
        else:
            raise Exception('Unknown line type: \'%s\'' % line_type)


class CallStack(object):

    def __init__(self, initial_call_stack_builder):
        self.count = 1
        self.signature = initial_call_stack_builder.signature
        self.lines = initial_call_stack_builder.lines
        for line in self.lines:
            for key, val in line.times.items():
                line.times[key] = [val]

    def add(self, call_stack_builder):
        assert self.signature == call_stack_builder.signature
        self.count += 1
        assert len(self.lines) == len(call_stack_builder.lines)
        for lsum, line in zip(self.lines, call_stack_builder.lines):
            assert lsum.tag == line.tag
            assert lsum.times.keys() == line.times.keys()
            for k, lst in lsum.times.items():
                lst.append(line.times[k])

    def finish(self):
        for line in self.lines:
            for lst in line.times.values():
                lst.sort()


builder = collections.defaultdict(CallStackBuilder)
call_stacks = collections.defaultdict(CallStack)

lines = 0
start = time.time()
with open(args.source) as f:
    for line in f:
        lines += 1
        inf = json.loads(line)
        thd = inf['thd']
        cs = builder[thd]
        if cs.add(inf):
            if cs.signature in call_stacks:
                call_stacks[cs.signature].add(cs)
            else:
                call_stacks[cs.signature] = CallStack(cs)
            del builder[thd]
time_taken = time.time() - start

call_stacks = sorted(call_stacks.values(),
                     key=lambda cs: cs.count,
                     reverse=True)
total_stacks = 0
for cs in call_stacks:
    total_stacks += cs.count
    cs.finish()


def percentile(N, percent, key=lambda x: x):
    """
    Find the percentile of an already sorted list of values.

    @parameter N - is a list of values. MUST be already sorted.
    @parameter percent - a float value from [0.0,1.0].
    @parameter key - optional key function to compute value from each element of N.

    @return - the percentile of the values
    """
    if not N:
        return None
    float_idx = (len(N) - 1) * percent
    idx = int(float_idx)
    result = key(N[idx])
    if idx < len(N) - 1:
        # interpolate with the next element's value
        result += (float_idx - idx) * (key(N[idx + 1]) - key(N[idx]))
    return result


def tidy_tag(tag):
    if tag[0:10] == 'GRPC_PTAG_':
        return tag[10:]
    return tag


def time_string(values):
    num_values = len(values)
    return '%.1f/%.1f/%.1f' % (1e6 * percentile(values, 0.5), 1e6 * percentile(
        values, 0.9), 1e6 * percentile(values, 0.99))


def time_format(idx):

    def ent(line, idx=idx):
        if idx in line.times:
            return time_string(line.times[idx])
        return ''

    return ent


BANNER = {'simple': 'Count: %(count)d', 'html': '<h1>Count: %(count)d</h1>'}

FORMAT = [
    ('TAG', lambda line: '..' * line.indent + tidy_tag(line.tag)),
    ('LOC', lambda line: '%s:%d' %
     (line.filename[line.filename.rfind('/') + 1:], line.fileline)),
    ('IMP', lambda line: '*' if line.important else ''),
    ('FROM_IMP', time_format(TIME_FROM_LAST_IMPORTANT)),
    ('FROM_STACK_START', time_format(TIME_FROM_STACK_START)),
    ('SELF', time_format(SELF_TIME)),
    ('TO_STACK_END', time_format(TIME_TO_STACK_END)),
    ('FROM_SCOPE_START', time_format(TIME_FROM_SCOPE_START)),
    ('SELF', time_format(SELF_TIME)),
    ('TO_SCOPE_END', time_format(TIME_TO_SCOPE_END)),
]

out = sys.stdout
if args.out != '-':
    out = open(args.out, 'w')

if args.fmt == 'html':
    # print >> out, '<html>'
    # print >> out, '<head>'
    # print >> out, '<title>Profile Report</title>'
    # print >> out, '</head>'
    print('<html>', file=out)
    print('<head>', file=out)
    print('<title>Profile Report</title>', file=out)
    print('</head>', file=out)

accounted_for = 0
for cs in call_stacks:
    print('\n', file=out)
    if args.fmt in BANNER:
        print(BANNER[args.fmt] % {
            'count': cs.count,
        }, file=out)

        # print >> out, BANNER[args.fmt] % {
            # 'count': cs.count,
        # }
    header, _ = zip(*FORMAT)
    table = []
    for line in cs.lines:
        fields = []
        for _, fn in FORMAT:
            fields.append(fn(line))
        table.append(fields)
    print(tabulate.tabulate(table, header, tablefmt=args.fmt), file=out)
    # print >> out, tabulate.tabulate(table, header, tablefmt=args.fmt)
    accounted_for += cs.count
    if accounted_for > .99 * total_stacks:
        break

if args.fmt == 'html':
    print ('</html>')

原文链接：https://blog.csdn.net/genius_002/article/details/107499053

一 交叉编译

二 执行测试用例

3. 编译测试用例

三 执行

四 性能profile

一交叉编译

二执行测试用例

三执行

四性能profile