Correlator-beamformer plugin

The CBF plugin provides GPU-accelerated correlation and beamforming for antenna arrays. It ingests channelized complex-voltage data, applies delay and complex gain models, forms voltage or detected beams, computes visibilities, and sends products to dedicated writers.

This page was verified against the supplied Galaxy collection version 1.0.0, whose collection name is EDD.edd_cbf. The repository also contains several calibration and controller components that are under active development; test the exact pinned revision as a coordinated system.

Processing graph

channelizer / F-engine
          |
          v
    cbf_pipeline <---- delay_manager
       |   |   | <--- gain / weight calibration
       |   |   |
       |   |   +---- visibility_writer
       |   +-------- power_beam_writer
       +------------ downstream voltage-beam consumer

Main roles

CBF roles
Role	Default product	Function
`EDD.edd_cbf.cbf_pipeline`	`beamformer`	Streaming GPU correlator and beamformer.
`EDD.edd_cbf.power_beam_writer`	`power_beam_writer`	Measuring pipeline that writes detected beam filterbanks.
`EDD.edd_cbf.visibility_writer`	`visibility_writer`	Measuring visibility formatter and writer.
`EDD.edd_cbf.channelizer_recorder`	`f_recorder`	Records selected channelized antenna voltages.
`EDD.edd_cbf.channelizer_controller`	`channelizer_controller`	Controls FPGA channelizers through a KATCP proxy.
`EDD.edd_cbf.delay_manager`	`delay_manager`	Publishes geometric and instrumental delay models.
`EDD.edd_cbf.complex_gain_calibrator`	`complex_gain_calibrator`	Produces or manages complex-gain solutions.
`EDD.edd_cbf.weight_generator`	`weight_generator`	Generates beam weights for phased-array operation.
`EDD.edd_cbf.acmwriter`	`edd_paf_acmwriter`	Writes autocorrelation matrices for calibration.
`EDD.edd_cbf.edd_cbf` / `installer`	none	Build and installation roles, not observing products.

Input and output formats

The supplied default input is MPIFR_EDD_ARGOS_Packetizer:1: channelized complex voltage with timestamp, antenna, channel, and polarization encoded in the SPEAD index vector. A complete descriptor includes total and per-stream channel count, group ID, sample rate, bit depth, samples per heap, central frequency, polarizations, sync time, and index-vector bit allocation.

The main pipeline can publish:

VoltageBeamDataStreamFormat:1: Complex tied-array voltage beams for coherent downstream processing.
PowerBeamDataStreamFormat:1: Detected beam spectra consumed by the power-beam writer.
FullStokesBeamDataStreamFormat:1: Full-Stokes detected beam products.
VisibilityDataStreamFormat:1: Cross- and autocorrelation products consumed by the visibility writer.

Only enable products with provisioned consumers or intentional external subscribers. Every enabled product contributes network, memory, and GPU load.

Main pipeline configuration

Correlator-beamformer settings
Key	Default	Meaning
`n_beams`	4	Number of processed beams.
`n_raw_beams`	2	Voltage beams retained before detection.
`nfft`	32	Fine-channelization transform length.
`numerator` / `denumerator`	1 / 1	Rational sampling or processing-rate factors. The key is spelled `denumerator` in the current implementation.
`nacc_spectra`	128	Beam-spectrum accumulation.
`nacc_visibilties`	`auto`	Visibility accumulation. The current key retains this spelling.
`antennas`	0 through 15	Antenna IDs selected from the input index vector.
`polarisations`	`[0, 1]`	Selected polarization IDs.
`output_products`	nested mapping	Enable and route voltage, power, full-Stokes, and visibility products.
`rx_network_backend`	`ibverbs`	Receive backend: `udp`, `ibverbs`, or `dpdk`.
`tx_network_backend`	`ibverbs`	Transmit backend: `udp`, `ibverbs`, or `dpdk`.
`n_capture_threads`	8	CPU threads used for input capture.
`n_dada_slots`	8	DADA ring-buffer slots.
`delay_channel`	`delay`	Redis pub/sub channel containing delay models.
`delay_timeout_interval`	10 s	Maximum wait for usable delay information.
`input_type`	`dummy`	Set to `network` for observation. The default is deliberately not a valid production assumption.
`nonfatal_numacheck`	false	Development-only escape hatch for unsuitable resource topology.

DPDK settings under dpdk_config.dpdk_eal_args are merged over the core defaults. They must agree with device binding and huge-page configuration in the host inventory.

Writer configuration

The power-beam writer selects beam IDs with beams, controls file size with nspectra_per_file, obtains per-beam pointing metadata from beam_metadata_channel, and uses base_output_directory below its pipeline data scope. Its input must include beam, subband, sample-rate, frequency, bandwidth, time-resolution, and bit-depth dimensions.

The visibility writer controls frequency-window grouping with channels_per_window and parallelism with formatter_nthreads and writer_nthreads. Validate the output format and metadata with downstream calibration or imaging software before commissioning.

Resource planning

CBF performance depends on antenna count, polarization count, channels, beams, correlation integration, enabled products, network backend, and GPU type. Before deploying:

Calculate input and every enabled output rate.
Benchmark the exact GPU architecture and CUDA suite.
Place NIC, GPU, CPU cores, and memory on a compatible NUMA node.
Reserve unique POSIX message queues and shared-memory keys.
Confirm delay and gain updates arrive within their timeout.
Confirm writer storage sustains the aggregate rate with adequate margin.

Monitoring and diagnosis

The main and writer roles install pipeline, receiver, and DADA panels. Monitor packet loss, ring-buffer fill, processing latency, delay age, scaling or statistics, writer queues, file progress, and output rate. A healthy receiver does not prove scientifically valid output: verify antenna ordering, polarization convention, delay sign, phase, scaling, and sky coordinates with a known source.

Testing

The repository contains extensive CUDA unit tests for beamforming, correlation, weights, scaling, statistics, transposition, and writers, plus Python tests for models, adapters, stream handlers, resources, and pipeline state transitions. Commissioning must add an end-to-end array test with known delays and complex gains and independently validate both beam and visibility products.