tcpdp is TCP dump tool with custom dumper and structured logger written in Go.
MIT License
tcpdp is TCP dump tool with custom dumper and structured logger written in Go.
tcpdp
has 3 modes:
tcpdp proxy
: TCP proxy server mode$ tcpdp proxy -l localhost:12345 -r localhost:1234 -d hex # hex.Dump()
$ tcpdp proxy -l localhost:55432 -r db.internal.example.com:5432 -d pg # Dump query of PostgreSQL
$ tcpdp proxy -l localhost:33306 -r db.example.com:3306 -d mysql # Dump query of MySQL
https://github.com/lestrrat-go/server-starter
$ start_server --port 33306 -- tcpdp proxy -s -r db.example.com:3306 -d mysql
$ tcpdp proxy -c config.toml
tcpdp probe
: Probe mode (like tcpdump)$ tcpdp probe -i lo0 -t localhost:3306 -d mysql # is almost the same setting as 'tcpdump -i lo0 host 127.0.0.1 and tcp port 3306'
$ tcpdp probe -i eth0 -t 3306 -d hex # is almost the same setting as 'tcpdump -i eth0 tcp port 3306'
tcpdp read
: Read pcap file mode$ tcpdump -i eth0 host 127.0.0.1 and tcp port 3306 -w mysql.pcap
$ tcpdp read mysql.pcap -d mysql -t 3306 -f ltsv
tcpdp config
Create config$ tcpdp config > myconfig.toml
$ tcpdp config
[tcpdp]
pidfile = "/var/run/tcpdp.pid"
dumper = "mysql"
[probe]
target = "db.example.com:3306"
interface = "en0"
bufferSize = "2MB"
immediateMode = false
snapshotLength = "auto"
internalBufferLength = 10000
filter = ""
[proxy]
useServerStarter = false
listenAddr = "localhost:3306"
remoteAddr = "db.example.com:3306"
[log]
dir = "/var/log/tcpdp"
enable = true
enableInternal = true
stdout = true
format = "ltsv"
rotateEnable = true
rotationTime = "daily"
rotationCount = 7
# You can execute arbitrary commands after rotate
# $1 = prev filename
# $2 = current filename
rotationHook = "/path/to/after_rotate.sh"
fileName = "tcpdp.log"
[dumpLog]
dir = "/var/log/dump"
enable = true
stdout = false
format = "json"
rotateEnable = true
rotationTime = "hourly"
rotationCount = 24
fileName = "dump.log"
$ go get github.com/k1LoW/tcpdp
client_addr
^
| tcpdp
+----------|---------------+
| v |
| proxy_listen_addr |
| + ^ |
| | | +--------+ |
| |<----+ dumper | |
| | |<--+ | |
| | | +--------+ |
| v + |
| proxy_client_addr |
| ^ |
+----------|---------------+
|
v
remote_addr
server
+--------------------------+
| |
| +---+---+
| <--------------| eth0 |----------->
| interface +---+---+
| /target ^ |
| | |
| tcpdp | |
| +--------+ | |
| | dumper +------+ |
| +--------+ |
+--------------------------+
tcpdp
+--------+ STDIN +--------+ STDOUT
| *.pcap +------>+ dumper +-------->
+--------+ +--------+
tcpdp proxy
or tcpdp probe
)key | description | mode |
---|---|---|
ts | timestamp | proxy / probe / read |
level | log level | proxy / probe |
msg | log message | proxy / probe |
error | error info | proxy / probe |
caller | error caller | proxy / probe |
conn_id | TCP connection ID by tcpdp | proxy / probe |
target | probe target | proxy / probe |
dumper | dumper type | proxy / probe |
use_server_starter | use server_starter | proxy |
conn_seq_num | TCP comunication sequence number by tcpdp | proxy |
client_addr | client address | tcpdp.log, hex, mysql, pg |
remote_addr | remote address | proxy |
proxy_listen_addr | listen address | proxy |
direction | client to remote: -> / remote to client: <-
|
proxy |
interface | probe target interface | probe |
mtu | interface MTU (Maximum Transmission Unit) | probe |
mss | TCP connection MSS (Max Segment Size) | probe |
probe_target_addr | probe target address | probe |
filter | BPF (Berkeley Packet Filter) | probe |
buffer_size | libpcap buffer_size | probe |
immediate_mode | libpcap immediate_mode | probe |
snapshot_length | libpcap snapshot length | probe |
internal_buffer_length | tcpdp internal packet buffer length | probe |
MySQL query dumper
NOTICE: MySQL query dumper require --target
option when tcpdp proxy
tcpdp probe
key | description | mode |
---|---|---|
ts | timestamp | proxy / probe / read |
conn_id | TCP connection ID by tcpdp | proxy / probe / read |
conn_seq_num | TCP comunication sequence number by tcpdp | proxy |
client_addr | client address | proxy |
proxy_listen_addr | listen address | proxy |
proxy_client_addr | proxy client address | proxy |
remote_addr | remote address | proxy |
direction | client to remote: -> / remote to client: <-
|
proxy |
interface | probe target interface | probe |
src_addr | src address | probe / read |
dst_addr | dst address | probe / read |
probe_target_addr | probe target address | probe |
proxy_protocol_src_addr | proxy protocol src address | probe / proxy /read |
proxy_protocol_dst_addr | proxy protocol dst address | probe / proxy /read |
query | SQL query | proxy / probe / read |
stmt_id | statement id | proxy / probe / read |
stmt_prepare_query | prepared statement query | proxy / probe / read |
stmt_execute_values | prepared statement execute values | proxy / probe / read |
character_set | character set | proxy / probe / read |
username | username | proxy / probe / read |
database | database | proxy / probe / read |
seq_num | sequence number by MySQL | proxy / probe / read |
command_id | command_id for MySQL | proxy / probe / read |
PostgreSQL query dumper
NOTICE: PostgreSQL query dumper require --target
option tcpdp proxy
tcpdp probe
key | description | mode |
---|---|---|
ts | timestamp | proxy / probe / read |
conn_id | TCP connection ID by tcpdp | proxy / probe / read |
conn_seq_num | TCP comunication sequence number by tcpdp | proxy |
client_addr | client address | proxy |
proxy_listen_addr | listen address | proxy |
proxy_client_addr | proxy client address | proxy |
remote_addr | remote address | proxy |
direction | client to remote: -> / remote to client: <-
|
proxy |
interface | probe target interface | probe |
src_addr | src address | probe / read |
dst_addr | dst address | probe / read |
probe_target_addr | probe target address | probe |
proxy_protocol_src_addr | proxy protocol src address | probe / proxy /read |
proxy_protocol_dst_addr | proxy protocol dst address | probe / proxy /read |
query | SQL query | proxy / probe / read |
portal_name | portal Name | proxy / probe / read |
stmt_name | prepared statement name | proxy / probe / read |
parse_query | prepared statement query | proxy / probe / read |
bind_values | prepared statement bind(execute) values | proxy / probe / read |
username | username | proxy / probe / read |
database | database | proxy / probe / read |
message_type | message type for PostgreSQL | proxy / probe / read |
key | description | mode |
---|---|---|
ts | timestamp | proxy / probe / read |
conn_id | TCP connection ID by tcpdp | proxy / probe / read |
conn_seq_num | TCP comunication sequence number by tcpdp | proxy |
client_addr | client address | proxy |
proxy_listen_addr | listen address | proxy |
proxy_client_addr | proxy client address | proxy |
remote_addr | remote address | proxy |
direction | client to remote: -> / remote to client: <-
|
proxy |
interface | probe target interface | probe |
src_addr | src address | probe / read |
dst_addr | dst address | probe / read |
probe_target_addr | probe target address | probe |
proxy_protocol_src_addr | proxy protocol src address | probe / proxy /read |
proxy_protocol_dst_addr | proxy protocol dst address | probe / proxy /read |
bytes | bytes string by hex.Dump | proxy / probe / read |
ascii | ascii string by hex.Dump | proxy / probe / read |