grpc-gateway

2019-05-16
#Golang #gRPC

grpc-gateway是Google protocol buffers系列化编译器protoc的插件。通过读取protobuf服务的定义，生成将RESTful json API请求转化为gRPC的反向代理。其通过protobuf中定义的google.api.http注释来生成。

grpc-gateway能同时提供gRPC和RESTful的APIs。

image

grpc-gateway架构图

grpc-gateway这个项目的目的是提供 HTTP+JSON 接口给你的gRPC服务。只需要在服务中加一点配置就能通过这个库产生反向代理。

安装

首先，需要本地安装了 protoc v3.00及以上。官方地址：

https://github.com/protocolbuffers/protobuf/releases

其次，需要安装grpc-gateway的包：

1
2
3
go get -u github.com/grpc-ecosystem/grpc-gateway/protoc-gen-grpc-gateway
go get -u github.com/grpc-ecosystem/grpc-gateway/protoc-gen-swagger
go get -u github.com/golang/protobuf/protoc-gen-go

下载完成后安装该项目，在$GOBIN下生成：

  • protoc-gen-grpc-gateway
  • protoc-gen-grpc-swagger
  • protoc-gen-go
  • 把$GOPATH加入到环境变量中

使用

我们可以新建一个grpc-gateway-demo项目

定义gRPC服务

首先，在grpc-gateway-demo目录下新建proto目录，在proto下新建your_service.proto:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
syntax = "proto3";
package example;

message Embedded {
    oneof mark {
        int64 progress = 1;
        string note = 2;
    }
}

message StringMessage {
    string id = 1;
    int64 num = 2;
    oneof code {
        int64 line_num = 3;
        string lang = 4;
    }
    Embedded status = 5;
    oneof ext {
        int64 en = 6;
        Embedded no = 7;
    }
}

service YourService {
  rpc Echo(StringMessage) returns (StringMessage) {}
}

以上是一个传统的proto文件，使用grpc-gateway需要对其进行修改：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
 syntax = "proto3";
 package example;
+
+import "google/api/annotations.proto";
+

message Embedded {
    oneof mark {
        int64 progress = 1;
        string note = 2;
    }
}

message StringMessage {
    string id = 1;
    int64 num = 2;
    oneof code {
        int64 line_num = 3;
        string lang = 4;
    }
    Embedded status = 5;
    oneof ext {
        int64 en = 6;
        Embedded no = 7;
    }
}

 service YourService {
-  rpc Echo(StringMessage) returns (StringMessage) {}
+  rpc Echo(StringMessage) returns (StringMessage) {
+    option (google.api.http) = {
+      post: "/v1/echo/{id}"
+      additional_bindings {
+        get: "/v1/echo/{id}/{num}"
+      }
+    };
+  }
 }

其中-表示删除部分，+表示增加部分

另外还需要在proto文件下新建google/api目录，在其中新增annotations.proto和http.proto文件。

http.proto文件可以拷贝：

https://github.com/googleapis/googleapis/blob/master/google/api/http.proto

annotations.proto文件可以拷贝：

https://github.com/googleapis/googleapis/blob/master/google/api/annotations.proto

其次，生成给序列化文件，windows下生成.pb.go文件：

1
protoc -IC:\Users\Leiying\go\bin -I.  -IC:\Users\Leiying\go\src  -IC:\Users\Leiying\go\src\github.com\grpc-ecosystem\grpc-gateway\third_party\googleapis  --go_out=plugins=grpc:.  your_service.proto

再生成反向代理，.pg.gw.go文件：

1
protoc -IC:\Users\Leiying\go\bin -I. -IC:\Users\Leiying\go\src  -IC:\Users\Leiying\go\src\github.com\grpc-ecosystem\grpc-gateway\third_party\googleapis   --grpc-gateway_out=logtostderr=true:.  your_service.proto

-I指定文件目录

执行完以上两步后生成：your_service.pb.go和 your_service.pb.gw.go

编写gateway服务

在grpc-gateway-demo目录下，新建cmd/example-grpc-gateway目录作为应用的入口

新建main.go文件：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
var (
	endpoint = flag.String("endpoint", "localhost:9090", "endpoint of the gRPC service")
	network = flag.String("network", "tcp", `one of "tcp" or "unix". Must be consistent to -endpoint`)
)

func main() {
	flag.Parse()
	defer glog.Flush()

	ctx := context.Background()
	opts := gateway.Options{
		Addr:":8080",
		GRPCServer:gateway.Endpoint{
			Network: *network,
			Addr: *endpoint,
		},
	}
	if err := gateway.Run(ctx, opts); err != nil {
		glog.Fatal(err)
	}
}

在grpc-gateway-demo目录下，新建gateway目录，编写gateway的主要逻辑，主要包括三个文件：main.go,gateway.go,handlers.go。

main.go:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
type Endpoint struct {
	Network, Addr string
}

type Options struct {
	Addr string
	GRPCServer Endpoint
	Mux []runtime.ServeMuxOption
}

func Run(ctx context.Context, opt Options) error {
	ctx , cancel := context.WithCancel(ctx)
	defer cancel()

	conn, err := dial(ctx, opt.GRPCServer.Network, opt.GRPCServer.Addr)
	if err != nil {
		return err
	}
	go func() {
		<- ctx.Done()
		if err := conn.Close(); err != nil {
			glog.Errorf("Failed to close a client connection to gRPC server: %v", err)
		}
	}()

	mux := http.NewServeMux()
	mux.HandleFunc("/healthz", healthzServer(conn))

	gw, err := newGateway(ctx, conn, opt.Mux)
	if err != nil {
		return err
	}
	mux.Handle("/", gw)

	s := &http.Server{
		Addr: opt.Addr,
		Handler: allowCORS(mux),
	}
	go func() {
		<- ctx.Done()
		glog.Infof("Shutting down the http server")
		if err := s.Shutdown(context.Background()); err != nil {
			glog.Errorf("Failed to shutdown http server: %v", err)
		}
	}()

	glog.Infof("Starting listening at %s", opt.Addr)
	if err := s.ListenAndServe(); err != http.ErrServerClosed {
		glog.Errorf("Failed to listen and serve: %v", err)
		return err
	}
	return nil
}

gateway.go:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
func newGateway(ctx context.Context, conn *grpc.ClientConn, opts []runtime.ServeMuxOption) (http.Handler, error) {
	mux := runtime.NewServeMux(opts...)

	for _, f := range []func(context.Context, *runtime.ServeMux, *grpc.ClientConn) error{
		proto.RegisterYourServiceHandler,
	} {
		if err := f(ctx, mux, conn);err != nil {
			return nil, err
		}
	}
	return mux, nil
}

func dial(ctx context.Context, network, addr string) (*grpc.ClientConn, error) {
	switch network {
	case "tcp":
		return dialTCP(ctx, addr)
	default:
		return nil, fmt.Errorf("unsupported network type %q", network)
	}
}

func dialTCP(ctx context.Context, addr string) (*grpc.ClientConn, error){
	return grpc.DialContext(ctx, addr, grpc.WithInsecure())
}

handlers.go:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
func allowCORS(h http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		if origin := r.Header.Get("Origin"); origin != "" {
			w.Header().Set("Access-Control-Allow-Origin", origin)
			if r.Method == "OPTIONS" && r.Header.Get("Access-Control-Request-Method") != ""{
				preflightHandler(w, r)
				return
			}
		}
		h.ServeHTTP(w, r)
	})
}

func preflightHandler(w http.ResponseWriter, r *http.Request) {
	headers := []string{"Content-Type", "Accept"}
	w.Header().Set("Access-Control-Allow-Headers", strings.Join(headers, ","))
	methods := []string{"GET", "HEAD", "POST", "PUT", "DELETE"}
	w.Header().Set("Access-Control-Allow-Methods", strings.Join(methods, ","))
	glog.Infof("preflight request for %s", r.URL.Path)
}

func healthzServer(conn *grpc.ClientConn) http.HandlerFunc{
	return func(w http.ResponseWriter, r *http.Request) {
		w.Header().Set("Content-Type", "text/plain")
		if s := conn.GetState(); s != connectivity.Ready {
			http.Error(w, fmt.Sprintf("grpc server is %s", s), http.StatusBadGateway)
			return
		}
		fmt.Fprintln(w, "ok")
	}
}

此时可以启动cmd/example-grpc-gateway下的main.go服务了，使用 postman 发送post localhost:8080/v1/echo/123会报错，原因在于8080端口的gateway服务会将请求转化为gRPC请求，转发到9090端口的gRPC服务，但目前9090端口没有gRPC服务，下面新建一个gRPC服务。

新建gRPC服务

在cmd下新建example-grpc-server目录，作为服务入口：

main.go:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
var (
	addr = flag.String("addr", ":9090", "endpoint of the gRPC service")
	network = flag.String("network", "tcp", "a valid network type which is consistent to addr")
)

func main() {
	flag.Parse()
	defer glog.Flush()

	ctx := context.Background()
	if err := server.Run(ctx, *network, *addr); err != nil {
		glog.Fatal(err)
	}
}

在grpc-gateway-demo下新建server目录，处理服务主逻辑，包含main.go，myserver.go两个文件。

main.go:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
func Run(ctx context.Context, network, addr string) error {
	l, err := net.Listen(network, addr)
	if err != nil {
		return err
	}
	defer func() {
		if err := l.Close(); err != nil {
			glog.Errorf("Failed to close %s %s: %v", network, addr, err)
		}
	}()

	s := grpc.NewServer()
	pb.RegisterYourServiceServer(s, newMyServer())

	go func() {
		defer s.GracefulStop()
		<- ctx.Done()
	}()
	return s.Serve(l)
}

myserver.go实现yourservice接口:

1
2
3
4
5
6
7
8
9
10
type myserver struct{}

func newMyServer() pb.YourServiceServer {
	return new(myserver)
}

func (myserver) Echo(ctx context.Context,msg *pb.StringMessage) (*pb.StringMessage, error) {
	glog.Info(msg)
	return msg, nil
}

启动服务，进行测试：

1
2
3
4
5
6
# Visit the apis
$ curl -XPOST http://localhost:8080/v1/echo/foo
{"id":"foo"}

$ curl  http://localhost:8080/v1/echo/foo/123
{"id":"foo","num":"123"}

参考

https://github.com/grpc-ecosystem/grpc-gateway