2013 10 03 NetFlow JiriTesar
2013 10 03 NetFlow JiriTesar
Technology Overview
Jiří Tesař, CCIE #14558
Systems Engineering
[email protected]
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 1
• Developed and patented at Cisco® Systems in 1996
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 2
IETF Scope
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 3
Monitoring
Feature Traditional Flexible Netflow
Netflow MMON ART
Need by AVC
Export
Protocol Netflow Netflow
IPFIX
Version 5 Version 9
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 4
Version Major Advantage Limits/Weaknesses
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 5
NetFlow v9 160+ fields to choose from including IPv6 and payload sections
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 6
NetFlow Enabled
Device
Traffic
Inspect
Packet NetFlow Cache
•Source IP address Flow Information Packets Bytes/packet
•Destination IP address
Address, ports… 11000 1528
•Source port …
•Destination port
•Layer 3 protocol
•TOS byte (DSCP) NetFlow
Create a flow from the packet Export Packets
•Input Interface
attributes
Cisco
NetFlow
Collector
Reporting
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 7
• Key fields are unique per flow
2 1 record 2 1
• Non-key fields are attributes or
Key Fields Packet 1 characteristics of a flow Key Fields Packet 2
Source IP 1.1.1.1 Source IP 3.3.3.3
Destination IP 2.2.2.2 • If packet key fields are unique, new Destination IP 4.4.4.4
Source port 23 entry in flow record is created Source port 80
Destination port 22078 Destination port 22079
• Otherwise, update the non-key
Layer 3 Protocol TCP - 6 Layer 3 Protocol TCP - 6
fields, i.e. packet count
TOS Byte 0 TOS Byte 0
Non-key Fields Packet 1 Non-key Fields Packet 2
Length 1250 Length 519
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 8
1. Create and update flows in NetFlow cache
Src Src Src Dst DstM Dst Bytes/
Srclf SrclPadd Dstlf DstlPadd Protocol TOS Flgs Pkts NextHop Active Idle
Port Msk AS Port sk AS Pkt
Fa1/0 173.100.21.2 Fa0/0 10.0.227.12 11 80 10 11000 00A2 /24 5 00A2 /24 15 10.0.23.2 1528 1745 4
Fa1/0 173.100.3.2 Fa0/0 10.0.227.12 6 40 0 2491 15 /26 196 15 /24 15 10.0.23.2 740 41.5 1
Fa1/0 173.100.20.2 Fa0/0 10.0.227.12 11 80 10 10000 00A1 /24 180 00A1 /24 15 10.0.23.2 1428 1145.5 3
Fa1/0 173.100.6.2 Fa0/0 10.0.227.12 6 40 0 2210 19 /30 180 19 /24 15 10.0.23.2 1040 24.5 14
3. Aggregation
ie: Protocol-port aggregation
4. Export version scheme becomes
Non-aggregated flows—export Version 5 or 9 Protocol Pkts SrcPort DstPort Bytes/Pkt
Header
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 9
Traffic
Flow Flow
Monitor 1 Monitor 2
TOS Byte 0
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 10
RFC3954
NetFlow v9
IPFIX
• There are “Template FlowSets”, which define the following data formats by a set of Field Types and their lengths, and
“Data FlowSets”, which are sets of actual statistics data
• The NetFlow v9 Exporter sends Templates separately (and less frequently) than data sets
• There are another sets of Template/Data FlowSets called “Option Template/Data FlowSets”, by which an Exporter can
send a Collector meta-data related to NetFlow process and so on.
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 11
Interface
Record “Y”
1
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 12
2
Peering Flows
Different Flow monitors for detecting different • Dest. AS
ISP
information: • Dest. Traffic Index
• BGP Next Hop
• DSCP
BRANCH
CAMPUS
IP Flows
• IP Subnets
Multicast Flows • Ports
Security Flows • Protocol
• Protocol • Protocol
• Ports • Interfaces
• Ports • Egress/Ingress
• IP Subnets • IP Addresses
• Packet Replication • TCP Flags
• Packet Section
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 13
Service Provider
Network Infrastructure Optimization
and Planning
Peering Arrangements
Traffic Engineering
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 14
Enterprise
User Monitoring/Profiling
Application Monitoring
CEF
QoS
DPI
Metadata
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 16
Integration
Link Layer Interface
Header NetFlow
ToS
NetFlow ü Monitors data in Layers 2 thru 4
Protocol ü Determines applications by combination of
IP Header Source Port or Port/IP Addressed
IP Address
ü Flow information who,
Destination IP
Address
what, when, where
Source NBAR
TCP/UDP Port
Header Destination ü Examines data from
Port Layers 3 thru 7
ü Utilizes Layers 3 and 4
plus packet inspection for classification
ü Stateful inspection of
Deep Packet
Data Packet (Payload) dynamic-port traffic
Inspection ü Packet and byte counts
NBAR
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 17
15.0(1)M
IOS XE 3.1S
News
First packet of a flow will create the Flow entry using the Key Fields”
Remaining packets of this flow will only update statistics (bytes,
© 2012 Cisco and/or its affiliates. All rights reserved. counters, timestamps) Cisco Confidential 18
enable users to aggregate on a
subset of the key and non-key fields
§ Top 4 IPv4 destinations sorted by
number of bytes:
# show flow monitor <monitor> Switch# show flow monitor <monitor>
aggregate ipv4 destination address
SrcIPadd DstIPadd TOS pkts bytes sort counter bytes top 4
10.1.0.5 172.16.10.19 0x00 1 64
DesIPadd flows bytes pkts
10.1.0.5 172.16.0.20 0x00 10 800
172.16.10.2 12 1358370 6708
10.1.0.95 172.16.10.19 0x00 200 16000 172.16.10.19 2 44640 1116
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 20
StealthWatch Reputation Feed
Management (Optional)
Console
StealthWatch Labs
Other tools/ Information Center
collectors
Cisco ISE
StealthWatch StealthWatch
FlowReplicator FlowCollector
NetFlow
NetFlo
w
NBAR NSEL
StealthWatch StealthWatch
FlowSensor FlowSensor VE
Users/Devices
Cisco Network
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 21
Who is 10.10.101.89?
Desktops Jan 3, 2013 Suspect Data 10.10.101.89 Atlanta, Multiple Hosts Observed 5.33G bytes.
& Trusted Loss Desktops Policy maximum allows up
Wireless to 500M bytes.
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 22
Policy Start Active Alarm Source Source Host Source User Device Target
Time Groups Name Type
Desktops Jan 3, 2013 Suspect Data 10.10.101.89 Atlanta, John Chambers Apple-iPad Multiple
& Trusted Loss Desktops Hosts
Wireless
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 23
• Flow Action field can provide additional context
• NAT stitching
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 24
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 25
• NetFlow is a mature Cisco IOS feature (in Cisco IOS since 1996)
• NetFlow provides input for accounting, performance, security, application
visibility, and billing applications
• Cisco standardizes on NetFlow/IPFIX: NetFlow 9 and Flexible NetFlow
consistency across many devices, including in hardware now
NetFlow v9 eases the exporting of additional fields
Flexible NetFlow is a major enhancement
2
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 26
6
TNF = Traditional (Non-Flexible) Netflow and FNF = Flexible Netflow
Enterprise & aggregation/edge Core
Release 12.0S/IOS-XR
Cisco IOS Software Release 12.2S
FNF
TNF TNF TNF FNF FNF
TNF
FNF
TNF FNF
TNF FNF
Cisco 12000 ASR9000
Catalyst 6K
Cisco 4500 Cisco Catalyst 6K Sup2T Series CRS-1
Cisco 4500 < Sup2T ASIC CRS-3
Cisco 7x00 ASR1000 Sup7 <= Sup5 7600 Series
QFP based ASIC
Series
NO FNF support Hardware limitation
Access DataCenter
Cisco IOS Software Releases FNF
TNF FNF Catalyst 29xx
TNF FNF Catalyst 3750
TNF FNF FNF
TNF FNF NO FNF support
TNF FNF Hardware limitation FNF
TNF FNF Cisco 7200/ Cat 6K
Cisco 2800 ASR1000 FNF
Cisco 3800 7300 Series QFP based
Sup2T
Cisco 1800 2900
3900 Catalyst 3750X Nexus 7000
Cisco 800 1900 Series
Series Next Gen Cat3K
Series Series
Nexus 1000V
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 27
For Your
Reference
• Ingress & Egress NetFlow. Useful for example to track packets de-
capsulated after tunneling mechanisms
• TCP Flags are now exported as part of the flow information. Very useful to
understand TCP flow directions and to detect denial of service attacks
• Export version 5 (the most used) and export version 9 (the most flexible)
are both supported
• Hitless ISSU
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 28
For Your
Reference
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 29
For Your
Reference
• TCP Flags are now exported as part of the flow information. Very useful
to understand TCP flow directions and to detect denial of service
attacks
• IPv4, IPv6 and Layer 2 Flows support
• Export version 5 (the most used) and export version 9 (the most
flexible) are both supported
• VRF aware export
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 30
For Your
Reference
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 31
For Your
Reference
3KX-SM-10G Service Module • System Scalability. Up to ~32K cached flows
Both
Full flow accounting • Sampled NetFlow. Effective hardware-based sampling to
and improve and preserve NetFlow table utilization
sampled NetFlow accounting • Ingress & Egress NetFlow
are supported
• Per Interface, Sub-Interface or Vlan activation
• Export version 9
• 3KX-SM-10G
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 32
For Your
Reference
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 33
NetFlow in Hardware on Catalyst 3850 New
Platform
• System Scalability NetFlow V4 / V6
Both
WS-C3850-24: up to 24k /12k cached Flows
Full flow accounting
and WS-C3850-48: up to 48k / 24k cached Flows
sampled NetFlow accounting • Sampled NetFlow. Effective hardware-based sampling to improve and
are supported preserve NetFlow table utilization
• Key Fields: IP address, MAC address, TOS, TCP Flags, and VLan
• Export version 9
• Only NetFlow Version 9 is supported for NetFlow exporter using the export-
protocol command option.
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 35
NetFlow in Hardware on cat 5760 New
Platform
• System Scalability: NetFlow v4/V6 Up to ~72K / 36k cached
flows
• Sampled NetFlow. Effective hardware-based sampling to
improve and preserve NetFlow table utilization
• Ingress & Egress NetFlow
• Export version 9
•
• NetFlow
http://www.cisco.com/go/netflow
3
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 37
7
Thank you.