Version: Next

1. Overview

1.1 Plugin Name#

Logging-RabbitMQ Plugin

1.2 Appropriate Scenario#

collect http request log to rabbitmq, consume rabbitmq message to another application and analysis.

1.3 Plugin functionality#

Apache ShenYu The gateway receives requests from the client, forwards them to the server, and returns the server results to the client. The gateway can record the details of each request，
The list includes: request time, request parameters, request path, response result, response status code, time consumption, upstream IP, exception information waiting.
The Logging-RabbitMQ plugin is a plugin that records access logs and sends them to the RabbitMQ cluster.

1.4 Plugin code#

Core Module shenyu-plugin-logging-rabbitmq.
Core Class org.apache.shenyu.plugin.logging.rabbitmq.LoggingRabbitmqPlugin
Core Claas org.apache.shenyu.plugin.logging.rabbitmq.client.RabbitmqLogCollectClient

1.5 Added Since Which shenyu version#

Since ShenYu 2.5.0

1.6 Technical Solutions#

Architecture Diagram

Full asynchronous collection and delivery of Logging inside the Apache ShenYu gateway
Logging platform by consuming the logs in the RabbitMQ cluster for repository, and then using Grafana, Kibana or other visualization platform to display

2. How to use plugin

2.1 Plugin-use procedure chart#

2.2 Import pom#

Add the Logging-RabbitMQ dependency to the gateway's pom.xml file.

<!--shenyu logging-rabbitmq plugin start--><dependency>    <groupId>org.apache.shenyu</groupId>    <artifactId>shenyu-spring-boot-starter-plugin-logging-rabbitmq</artifactId>    <version>${project.version}</version></dependency><!--shenyu logging-rabbitmq plugin end-->

2.3 Enable plugin#

In shenyu-admin --> Basic Configuration --> Plugin Management --> loggingRabbitMQ, configure the rabbitMQ parameter and set it to on.

2.4 Config plugin#

2.4.1 Open the plugin and configure rabbitmq, configure it as follows.#

The individual configuration items are described as follows:

config-item	type	description	remarks
host	type	IP	必须
port	type	PORT	必须
username	String	rabbitmq username	可选
password	String	rabbitmq password	可选
virtualHost	String	rabbitmq virtualHost	必须，默认/
exchangeType	String	rabbitmq exchange type	必须，默认direct
exchangeName	String	rabbitmq exchange name	必须
queueName	String	rabbitmq queue name	必须
routingKey	String	rabbitmq routing key	必须
durable	Boolean	message durable	必须，默认true
exclusive	Boolean	message exclusive	必须，默认false
autoDelete	String	message auto delete	必须，默认false
args	String	rabbitmq args，exaple：{"x-delay":"1000"}，delay queue	可选
sampleRate	String	Sampling rate, range 0~1, 0: off, 0.01: acquisition 1%, 1: acquisition 100%	Optional, default 1, all collection
maxResponseBody	Ingeter	Maximum response size, above the threshold no response will be collected	Optional, default 512KB
maxRequestBody	Ingeter	Maximum request body size, above the threshold no request body will be collected	Optional, default 512KB

Except for host,port,virtualHost,exchangeType,exchangeName,queueName,routingKey,durable,exclusive,autoDelete, all others are optional.

2.4.2 Configuring Selectors and Rulers#

For detailed configuration of selectors and rules, please refer to: Selector and rule management。

In addition, sometimes a large gateway cluster corresponds to multiple applications (business), different applications (business) corresponds to different topics, related to isolation,
then you can configure different topics (optional) and sampling rate (optional) by selector, the meaning of the configuration items as shown in the table above.
The operation is shown below:

2.5 Logging Info#

collect request info as follows

Field Name	Meaning	Description
clientIp	Client IP
timeLocal	Request time string, format: yyyy-MM-dd HH:mm:ss.SSS
method	request method (different rpc type is not the same, http class for: get, post wait, rpc class for the interface name)
requestHeader	Request header (json format)
responseHeader	Response header (json format)
queryParams	Request query parameters
requestBody	Request Body (body of binary type will not be captured)
requestUri	Request uri
responseBody	Response body
responseContentLength	Response body size
rpcType	rpc type
status	response status
upstreamIp	Upstream (program providing the service) IP
upstreamResponseTime	Time taken by the upstream (program providing the service) to respond to the request (ms ms)
userAgent	Requested user agent
host	The requested host
module	Requested modules
path	The requested path
traceId	Requested Link Tracking ID	Need to access link tracking plugins, such as skywalking,zipkin

2.6 Examples#

2.6.1 Collect Http Log by RabbitMQ#

2.6.1.1 Plugin Configuration#

Open the plugin and configure rabbitmq, configure it as follows.

2.6.1.2 Selector Configuration#

For detailed configuration of selectors and rules, please refer to: Selector and rule management。

2.6.1.3 Rule Configuration#

2.6.1.4 Request Service#

2.6.1.5 Consumption and display of Logging#

As each logging platform has differences, such as storage available clickhouse, ElasticSearch, etc., visualization has self-developed or open source Grafana, Kibana, etc..
Logging-RabbitMQ plugin uses RabbitMQ to decouple production and consumption, while outputting logs in json format, consumption and visualization require users to choose different technology stacks to achieve their own situation.

2.6.1.6 Panel Display#

Users can choose to visualize the implementation according to their own situation.
The following shows the effect of Grafana:
Grafana Sandbox Experience

3. How to disable plugin

In shenyu-admin --> BasicConfig --> Plugin --> loggingRabbitMQ set Status disable.