Skip to content

leitwert-net/ftlbgp

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

6 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

ftlbgp

A Python parser for BGP data in MRT or looking glass format.

Key features

  • Support for all MRT entry types, BGP messages, and BGP attributes
  • Customizable record and attribute types (no parsing of unneeded data)
  • Programmatic data access with optional CSV and JSON serialization
  • Raw values and human-readable output (e.g. integers vs. strings)
  • Rapid prototyping (namedtuple) and high-performance mode (tuple)
  • Context manager with built-in statistics and flexible error handling
  • Zero-copy operations on all data items (as fast as it gets in Python)

Compatibility

This package is compatible with python3.6 and pypy3.6-v7.0.0 or greater.

Installation

Run pip install ftlbgp or

~$ git clone https://github.com/leitwert-net/ftlbgp.git` ~$ cd ftlbgp.git/src 

to download and run the source code manually.

Usage

Parsing MRT files

~$ python3 -m ftlbgp -h usage: python3 -m ftlbgp [-h] [--pkg-help] [--json] <FILE> [<FILE> ...] ftlbgp [v1.0.3] - Parse BGP archives in MRT or looking glass format positional arguments: <FILE> input file with BGP data (supports bz2/gz) optional arguments: -h, --help show this help message and exit --pkg-help show package help for BgpParser usage --json output BGP records in JSON format 

Programmatic use

# Import parser from ftlbgp import BgpParser # Prepare input file filename = ... # Parse default records and attributes with BgpParser() as parse: for record in parse(filename): print(record)

Customization

# Parse all records with BgpParser(bgp_records=BgpParser.bgp.records.ALL) as parse: for record in parse(filename): print(record) # Parse specific records (route and error) with BgpParser(bgp_records=BgpParser.bgp.records.route | BgpParser.bgp.records.error) as parse: for record in parse(filename): print(record) # Parse all route attributes with BgpParser(bgp_route=BgpParser.bgp.route.ALL) as parse: for record in parse(filename): print(record) # Parse specific route attributes (default and local_pref) with BgpParser(bgp_route=BgpParser.bgp.route.DEFAULT | BgpParser.bgp.route.local_pref) as parse: for record in parse(filename): print(record)

Full specification

BgpParser() This @FtlParser instance is used to read input files and generate a set of BGP records (Python tuples). It must be used with a context manager (see sample usage below) and accepts the following arguments. Keyword arguments: named_records - Return named tuples instead of plain unnamed tuple records. [Default: True] serialize - Convert output records to JSON (if named) or CSV (if unnamed). [Default: False] use_cache - Cache expensive low-entropy values (memory consumption < 1MB). [Default: True] raise_on_errors - Raise exceptions and stop parsing in case of data errors. [Default: False] bgp_records - Select the types of BgpRecord entries to be returned by the parser. Supports bitwise logical operators OR, AND, NOT to specify multiple records. [Default: BgpParser.bgp.records.route | BgpParser.bgp.records.stats | BgpParser.bgp.records.error Available: BgpParser.bgp.records.peer_table BgpParser.bgp.records.state_change BgpParser.bgp.records.keep_alive BgpParser.bgp.records.route_refresh BgpParser.bgp.records.notification BgpParser.bgp.records.open BgpParser.bgp.records.ALL BgpParser.bgp.records.NONE BgpParser.bgp.records.DEFAULT] bgp_peer_table - Select [optionally human-readable] attribute to be included in BgpPeerTableRecord entries. Supports bitwise logical operators (OR/AND/NOT) to specify multiple attributes. [Default: BgpParser.bgp.peer_table[.human].type | BgpParser.bgp.peer_table[.human].peer_protocol | BgpParser.bgp.peer_table[.human].peer_bgp_id | BgpParser.bgp.peer_table[.human].peer_as | BgpParser.bgp.peer_table[.human].peer_ip Available: BgpParser.bgp.peer_table[.human].collector_bgp_id BgpParser.bgp.peer_table[.human].view_name BgpParser.bgp.peer_table[.human].ALL BgpParser.bgp.peer_table[.human].NONE BgpParser.bgp.peer_table[.human].DEFAULT] bgp_state_change - Select [optionally human-readable] attribute to be included in BgpStateChangeRecord entries. Supports bitwise logical operators (OR/AND/NOT) to specify multiple attributes. [Default: BgpParser.bgp.state_change[.human].type | BgpParser.bgp.state_change[.human].timestamp | BgpParser.bgp.state_change[.human].peer_protocol | BgpParser.bgp.state_change[.human].peer_as | BgpParser.bgp.state_change[.human].peer_ip | BgpParser.bgp.state_change[.human].old_state | BgpParser.bgp.state_change[.human].new_state Available: BgpParser.bgp.state_change[.human].ALL BgpParser.bgp.state_change[.human].NONE BgpParser.bgp.state_change[.human].DEFAULT] bgp_route - Select [optionally human-readable] attribute to be included in BgpRouteRecord entries. Supports bitwise logical operators (OR/AND/NOT) to specify multiple attributes. [Default: BgpParser.bgp.route[.human].type | BgpParser.bgp.route[.human].source | BgpParser.bgp.route[.human].sequence | BgpParser.bgp.route[.human].timestamp | BgpParser.bgp.route[.human].peer_protocol | BgpParser.bgp.route[.human].peer_bgp_id | BgpParser.bgp.route[.human].peer_as | BgpParser.bgp.route[.human].peer_ip | BgpParser.bgp.route[.human].nexthop_protocol | BgpParser.bgp.route[.human].nexthop_ip | BgpParser.bgp.route[.human].prefix_protocol | BgpParser.bgp.route[.human].prefix | BgpParser.bgp.route[.human].path_id | BgpParser.bgp.route[.human].aspath | BgpParser.bgp.route[.human].origin | BgpParser.bgp.route[.human].communities | BgpParser.bgp.route[.human].large_communities Available: BgpParser.bgp.route[.human].extended_communities BgpParser.bgp.route[.human].multi_exit_disc BgpParser.bgp.route[.human].atomic_aggregate BgpParser.bgp.route[.human].aggregator_protocol BgpParser.bgp.route[.human].aggregator_as BgpParser.bgp.route[.human].aggregator_ip BgpParser.bgp.route[.human].only_to_customer BgpParser.bgp.route[.human].originator_id BgpParser.bgp.route[.human].cluster_list BgpParser.bgp.route[.human].local_pref BgpParser.bgp.route[.human].attr_set BgpParser.bgp.route[.human].as_pathlimit BgpParser.bgp.route[.human].aigp BgpParser.bgp.route[.human].attrs_unknown BgpParser.bgp.route[.human].ALL BgpParser.bgp.route[.human].NONE BgpParser.bgp.route[.human].DEFAULT] bgp_keep_alive - Select [optionally human-readable] attribute to be included in BgpKeepAliveRecord entries. Supports bitwise logical operators (OR/AND/NOT) to specify multiple attributes. [Default: BgpParser.bgp.keep_alive[.human].type | BgpParser.bgp.keep_alive[.human].timestamp | BgpParser.bgp.keep_alive[.human].peer_protocol | BgpParser.bgp.keep_alive[.human].peer_as | BgpParser.bgp.keep_alive[.human].peer_ip Available: BgpParser.bgp.keep_alive[.human].ALL BgpParser.bgp.keep_alive[.human].NONE BgpParser.bgp.keep_alive[.human].DEFAULT] bgp_route_refresh - Select [optionally human-readable] attribute to be included in BgpRouteRefreshRecord entries. Supports bitwise logical operators (OR/AND/NOT) to specify multiple attributes. [Default: BgpParser.bgp.route_refresh[.human].type | BgpParser.bgp.route_refresh[.human].timestamp | BgpParser.bgp.route_refresh[.human].peer_protocol | BgpParser.bgp.route_refresh[.human].peer_as | BgpParser.bgp.route_refresh[.human].peer_ip | BgpParser.bgp.route_refresh[.human].refresh_protocol Available: BgpParser.bgp.route_refresh[.human].ALL BgpParser.bgp.route_refresh[.human].NONE BgpParser.bgp.route_refresh[.human].DEFAULT] bgp_notification - Select [optionally human-readable] attribute to be included in BgpNotificationRecord entries. Supports bitwise logical operators (OR/AND/NOT) to specify multiple attributes. [Default: BgpParser.bgp.notification[.human].type | BgpParser.bgp.notification[.human].timestamp | BgpParser.bgp.notification[.human].peer_protocol | BgpParser.bgp.notification[.human].peer_as | BgpParser.bgp.notification[.human].peer_ip | BgpParser.bgp.notification[.human].error_code | BgpParser.bgp.notification[.human].error_subcode | BgpParser.bgp.notification[.human].data Available: BgpParser.bgp.notification[.human].ALL BgpParser.bgp.notification[.human].NONE BgpParser.bgp.notification[.human].DEFAULT] bgp_open - Select [optionally human-readable] attribute to be included in BgpOpenRecord entries. Supports bitwise logical operators (OR/AND/NOT) to specify multiple attributes. [Default: BgpParser.bgp.open[.human].type | BgpParser.bgp.open[.human].timestamp | BgpParser.bgp.open[.human].peer_protocol | BgpParser.bgp.open[.human].peer_as | BgpParser.bgp.open[.human].peer_ip | BgpParser.bgp.open[.human].version | BgpParser.bgp.open[.human].my_as | BgpParser.bgp.open[.human].hold_time | BgpParser.bgp.open[.human].bgp_id | BgpParser.bgp.open[.human].capabilities Available: BgpParser.bgp.open[.human].params_unknown BgpParser.bgp.open[.human].ALL BgpParser.bgp.open[.human].NONE BgpParser.bgp.open[.human].DEFAULT] bgp_stats - Select [optionally human-readable] attribute to be included in BgpStatsRecord entries. Supports bitwise logical operators (OR/AND/NOT) to specify multiple attributes. [Default: BgpParser.bgp.stats[.human].type | BgpParser.bgp.stats[.human].parser_lifetime | BgpParser.bgp.stats[.human].parser_runtime | BgpParser.bgp.stats[.human].parser_errors | BgpParser.bgp.stats[.human].lgl_runtime | BgpParser.bgp.stats[.human].lgl_entries | BgpParser.bgp.stats[.human].lgl_errors | BgpParser.bgp.stats[.human].mrt_runtime | BgpParser.bgp.stats[.human].mrt_entries | BgpParser.bgp.stats[.human].mrt_errors | BgpParser.bgp.stats[.human].mrt_fixes | BgpParser.bgp.stats[.human].mrt_bgp_entry_types | BgpParser.bgp.stats[.human].mrt_bgp_message_types | BgpParser.bgp.stats[.human].mrt_bgp_attribute_types | BgpParser.bgp.stats[.human].mrt_bgp_capability_types | BgpParser.bgp.stats[.human].bgp_routes_rib_ipv4 | BgpParser.bgp.stats[.human].bgp_routes_rib_ipv6 | BgpParser.bgp.stats[.human].bgp_routes_announce_ipv4 | BgpParser.bgp.stats[.human].bgp_routes_announce_ipv6 | BgpParser.bgp.stats[.human].bgp_routes_withdraw_ipv4 | BgpParser.bgp.stats[.human].bgp_routes_withdraw_ipv6 Available: BgpParser.bgp.stats[.human].ALL BgpParser.bgp.stats[.human].NONE BgpParser.bgp.stats[.human].DEFAULT] bgp_error - Select [optionally human-readable] attribute to be included in BgpErrorRecord entries. Supports bitwise logical operators (OR/AND/NOT) to specify multiple attributes. [Default: BgpParser.bgp.error[.human].type | BgpParser.bgp.error[.human].source | BgpParser.bgp.error[.human].scope | BgpParser.bgp.error[.human].record | BgpParser.bgp.error[.human].reason | BgpParser.bgp.error[.human].message | BgpParser.bgp.error[.human].data | BgpParser.bgp.error[.human].trace Available: BgpParser.bgp.error[.human].ALL BgpParser.bgp.error[.human].NONE BgpParser.bgp.error[.human].DEFAULT] Returns: parse(<filename>) - Parse function that accepts a filename as single positional argument and generates all specified BGP records on invocation. Input files can be provided in .bz2 or .gz format. The parse function may be invoked multiple times within a single context. Raises: FtlError - Generic parser or runtime error. FtlFileError - Failure during access of input file. FtlFormatError - Unexpected format of input file. FtlDataError - Invalid data entry in input file. 

Author

Johann SCHLAMP <schlamp@leitwert.net>

License

Copyright (C) 2014-2025 Leitwert GmbH

This software is distributed under the terms of the MIT license.
It can be found in the LICENSE file or at https://opensource.org/licenses/MIT.

About

A Python parser for BGP data in MRT or looking glass format.

Topics

Resources

License

Stars

Watchers

Forks

Languages