新一代Ntopng网络流量监控—可视化和架构分析

 

NTopng主要特性

多协议网络流量；IPv4/IPv6活跃主机

网络流量监控（RRD存储格式）；基于nDPI实现应用协议发现

做为 NetFlow/sFlow  采集器 (Cisco/ Juniper 路由器)  ；交换机配合 nProbe.

 

效果图

 

 

What ntopng can do for me? 

• http://www.ntop.org/products/ntop

• Sort network traffic according to many protocols

• Show network traffic and IPv4/v6 active hosts

• Store on disk persistent traffic statistics in RRD format

• Geolocate hosts

• Discover application protocols by leveraging on nDPI, ntop’s DPI framework.

• Characterise HTTP traffic by leveraging on characterisation services provided by block.si. ntopng comes with a demo characterisation key, but if you need a permanent one, please mail info@block.si.

• Show IP traffic distribution among the various protocols

• Analyse IP traffic and sort it according to the source/destination

• Display IP Traffic Subnet matrix (who’s talking to who?)

• Report IP protocol usage sorted by protocol type

• Act as a NetFlow/sFlow collector for flows generated by routers (e.g. Cisco and Juniper) or switches (e.g. Foundry Networks) when used together with nProbe.

• Produce HTML5/AJAX network traffic statistics

 

Ntopng 架构

 

Libpcap

网络数据包捕获函数包

Sqlite

轻型数据库，多语言支持（ntopng中应该是和python结合），不少嵌入式系统也用到它

Gdbm：DBM的GNU版本，使用hash存储非结构化数据

 Python

autoconf、automake、pkg-config、libtool（提供通用的库编译支持）

Gettext、icu4c：国际化(I18N)和本地化(L10N)，多语言支持

libffi：“FFI” 的全名是 Foreign Function Interface，一般指的是容许以一种语言编写的代码调用另外一种语言的代码。而 “Libffi” 库只提供了最底层的、与架构相关的、完整的”FFI”，所以在它之上必须有一层来负责管理两种语言之间参数的格式转换

Gobject-introspection：（简称 GI）用于产生与解析 C 程序库 API 元信息，以便于动态语言（或托管语言）绑定基于 C + GObject 的程序库

json-glib、json-c、openssl、glib

 

ZeroMQ

号称最快的消息库，协议级，目标是成为Linux的一部分。

《ZeroMQ社区》：《ZeroMQ社区生态白皮书》、《ZMQ架构哲学》

 

libtasn1：用于开发 ASN.1 (Abstract Syntax Notation One) 结构管理的 C 库

gmp

Nettle：a low-level cryptographic library （加密）

Gnutls：（加密）

libpng：the official PNG reference library （图形）

pixman：像素管理（图形）

Cairo：a2Dgraphicslibrarywithsupportformultipleoutputdevices.

Freetype：FreeType库是一个彻底免费（开源）的、高质量的且可移植的字体引擎，它提供统一的接口来访问多种字体格式文件，包括TrueType,OpenType, Type1, CID,CFF, Windows FON/FNT, X11 PCF等

fontconfig：字体库管理

Pango

Pango（Παν语）是一个开放源代码的自由函数库，用于高质量地渲染国际化的文字。Pango能够使用不一样的后端字体，并提供了跨平台支持。依赖Harfbuzz ：一个开源的text opentype layout 引擎。

RRDtool

源自MRTG（多路由器流量绘图器）。MRTG是有一个大学链接到互联网链路的使用率的小脚本开始的。MRTG后来被看成绘制其余数据源的工具使用，包括温度、速度、电压、输出量等等。

参考：http://blog.sina.com.cn/s/blog_4e424e2101000b5s.html

luajit

C语言写的Lua的解释器

 

Geoip：IP GIS图形

 Redis

Redis是一个开源的使用ANSIC语言编写、支持网络、可基于内存亦可持久化的日志型、Key-Value数据库，并提供多种语言的API。Ntopng的Redis数据结构以下：

Brew快速安装

yanruideMacBook-Pro:~ yanrui$ ruby -v

ruby 2.0.0p481 (2014-05-08 revision 45883) [universal.x86_64-darwin14]

yanruideMacBook-Pro:~ yanrui$ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

==> This script will install:

/usr/local/bin/brew

/usr/local/Library/...

/usr/local/share/man/man1/brew.1

Press RETURN to continue or any other key to abort

==> Downloading and installing Homebrew...

remote: Counting objects: 237423, done.

remote: Compressing objects: 100% (1040/1040), done.

remote: Total 237423 (delta 711), reused 0 (delta 0), pack-reused 236381

Receiving objects: 100% (237423/237423), 32.52 MiB | 1.01 MiB/s, done.

Resolving deltas: 100% (176649/176649), done.

From https://github.com/Homebrew/homebrew

* [new branch]      master     -> origin/master

HEAD is now at 0faf905 Return early for the == case in Version#<=>

==> Installation successful!

==> Next steps

Run `brew doctor` before you install anything

Run `brew help` to get started

yanruideMacBook-Pro:~ yanrui$brew install ntopng

cairo: XQuartz is required to install this formula.

You can install with Homebrew Cask:

brew install Caskroom/cask/xquartz

You can download from:

https://xquartz.macosforge.org

pango: XQuartz is required to install this formula.

You can install with Homebrew Cask:

brew install Caskroom/cask/xquartz

You can download from:

https://xquartz.macosforge.org

Error: Unsatisified requirements failed this build.

yanruideMacBook-Pro:~ yanrui$ brew install Caskroom/cask/xquartz

Cloning into '/usr/local/Library/Taps/caskroom/homebrew-cask'...

remote: Counting objects: 128670, done.

remote: Compressing objects: 100% (12/12), done.

remote: Total 128670 (delta 4), reused 0 (delta 0), pack-reused 128658

Receiving objects: 100% (128670/128670), 37.17 MiB | 6.00 KiB/s, done.

Resolving deltas: 100% (85113/85113), done.

Checking connectivity... done.

Ntopng 服务启动

yanruideMacBook-Pro:~ yanrui$ sudo ntopng

19/Mar/2015 11:51:40 [Ntop.cpp:586] Setting local networks to 192.168.1.0/24,0.0.0.0/32,224.0.0.0/8,239.0.0.0/8,255.255.255.255/32,127.0.0.0/8

19/Mar/2015 11:51:40 [Redis.cpp:74] Successfully connected to Redis 127.0.0.1:6379

19/Mar/2015 11:51:40 [PcapInterface.cpp:81] Reading packets from interface en0...

19/Mar/2015 11:51:40 [Ntop.cpp:710] Registered interface en0 [id: 0]

19/Mar/2015 11:51:40 [PcapInterface.cpp:81] Reading packets from interface awdl0...

19/Mar/2015 11:51:40 [Ntop.cpp:710] Registered interface awdl0 [id: 1]

19/Mar/2015 11:51:40 [PcapInterface.cpp:81] Reading packets from interface en1...

19/Mar/2015 11:51:40 [Ntop.cpp:710] Registered interface en1 [id: 2]

19/Mar/2015 11:51:40 [PcapInterface.cpp:81] Reading packets from interface en2...

19/Mar/2015 11:51:40 [Ntop.cpp:710] Registered interface en2 [id: 3]

19/Mar/2015 11:51:40 [PcapInterface.cpp:81] Reading packets from interface p2p0...

19/Mar/2015 11:51:40 [Ntop.cpp:710] Registered interface p2p0 [id: 4]

19/Mar/2015 11:51:40 [PcapInterface.cpp:81] Reading packets from interface lo0...

19/Mar/2015 11:51:40 [Ntop.cpp:710] Registered interface lo0 [id: 5]

19/Mar/2015 11:51:40 [Utils.cpp:251] User changed to nobody

19/Mar/2015 11:51:40 [main.cpp:184] PID stored in file /var/tmp/ntopng.pid

19/Mar/2015 11:51:40 [HTTPserver.cpp:392] HTTP server listening on port 3000

 

P2P演示案例

演示案例：

A－>B经过QQ传递一个文件，在B端开启监测服务。

在NTopng WEB 控制台能够实时观测到B端主机的当前流量变化、目标IP地址、协议等。

 

欢迎交流指正！

 

预备话题

如下话题构思准备中，请关注

1.NTop在服务器集群中的多点探测部署

2.插件：支持NetFlow

 

推荐电子书：《Linux Perf Master》

以Linux性能为核心，覆盖评估诊断、监控、优化工具、方法论和参考案例,欢迎订阅、下载、批评指正。 本书发表在GitBook平台: https://www.gitbook.com/book/riboseyim/linux-perf-master/details 

更多精彩内容扫码关注公众号：

RiboseYim's Blog：https://riboseyim.github.io