grpc_load_balancer is a Python package that allows you to easily configure and manage separate gRPC server and client instances using a metrics-based load balancing approach. This is useful when you have multiple services running and want to distribute the load efficiently among them. The package includes a connection forwarder, a metrics-based server finder, and a configuration loader. It implemetns the following design:
- Forward connections from clients to gRPC servers.
- Use Prometheus metrics to select the best server to handle a request.
- Configure and manage gRPC server finder using environment variables.
You can install grpc_load_balancer using pip:
pip install grpc_load_balancerInstall promethus-client:
pip install prometheus-clientHere's a simple example of how to use grpc_load_balancer with a gRPC server and prometeus-client:
import grpc from grpc_load_balancer import GrpcConnnectionForwarder from prometheus_client import start_http_server, Gauge grpc_server = create_example_grpc_server() # implement this function yourself connection_counter = Gauge('connections_num', 'Number of connections forwarded') forwarder = GrpcConnnectionForwarder( grpc_server, callback=lambda value: connection_counter.set(value) ) start_http_server(prometheus_port) # forwarder_port - port to listen connections from gRPC clients # Note this call is blocking forwarder.serve(host="0.0.0.0", port=forwarder_port) # If you want to run the server in a separate thread, and want to stop it later, use the forwarder.stop() method import grpc from grpc_load_balancer import EnvConfigLoader, MetricsBasedServerFinder# This is example how your environment variables can look like # os.environ['LB_METRICS_NAME'] = 'connections_num' # os.environ['LB_HOST1_HOST'] = 'localhost' # os.environ['LB_HOST1_PORT'] = '20000' # os.environ['LB_HOST1_METRICS_PORT'] = '20001' # os.environ['LB_HOST2_HOST'] = 'localhost' # os.environ['LB_HOST2_PORT'] = '30000' # os.environ['LB_HOST2_METRICS_PORT'] = '30001' env_config_loader = EnvConfigLoader(prefix="LB") metrics_based_server_finder = env_config_loader.init_metrics_based_server_finder() best_host = metrics_based_server_finder.fetch_metrics() channel = grpc.insecure_channel(f"{best_host[0]}:{best_host[1]}") client = create_example_grpc_client(channel) # implement this function yourself # Use the client to send requests to the serverIf you have any problems with grpc_load_balancer, please open an issue on GitHub. Also, you can check tests/test_grpc_load_balancer.py for more information.
We welcome contributions to grpc_load_balancer. If you find a bug or want to propose a new feature, please open a GitHub issue or submit a pull request.
This project is licensed under the MIT License. See LICENSE for details.