User Guide

KM3Pipe has two main workflows: An Event-by-event processing approach, and a 2D-table/array approach.

The event-by-event approach is handled by the km3pipe Pipeline, where you chain together different modules for processing events (e.g. Hit cleaning, reconstruction), and then pump events – also referred to as blobs – down the pipeline.

The table-based approach is using the python libraries numpy and pandas from the SciPy ecosystem, which are fast and easy tools for scientific computation and data analysis, based on a clean tabular data format.

The preferred on-disk file format is HDF5, although km3pipe can read all ROOT/Text based km3net file format like Jpp, Aanet, Evt, etc.

• Pipeline Workflow
  • Modules
  • Pumps / Sinks
  • Logging and Printing
  • Configuring the Pipeline using Configuration Files
• Data Structures
  • Hits and McHits
  • Calibrating Hits and McHits
  • Sorting of Hits
• Pipeline Features
  • Introduction
  • Ctrl+C for Graceful Exits
  • Time / Memory / CPU Usage
  • Services
  • Automatic HDF5 Writing
• Storing Data
• HDF5
  • Data Hierarchy
    • (Experts) Data Substructure
  • Conversion Utils
• Command Line Tools
  • KM3Pipe
    • update
    • runtable
    • triggersetup
    • triggermap
  • DataBase
    • streamds
  • PipeInspector
  • HDF5 CLI Utils
    • tohdf
    • calibrate
    • hdf2root
    • h5info
    • h5tree
    • h5tree
    • ptdump (from PyTables)
    • pttree (from PyTables)
    • km3h5concat
• DBManager
  • Dataformats
  • DB Authentication
  • Retrieving the list of detectors
  • CLBs
  • Fun with DOMs
  • Datalogs
    • Parameters
    • Parameter Units and Value Conversions
    • Retrieving Parameter Data
  • StreamDS
  • The streamds CLI
  • Uploading “runsummarynumbers”
• FAQ
  • AAnet formats (tohdf5 --aa-format=...)
  • AanetPump/tohdf5 segfaults: