\documentclass{ivs2e}       % for LaTeX2e class file
\usepackage{epsfig}         % for including .ps figure files
\usepackage{html}
\usepackage{url}
%
%
\Title{The Bonn Astro/Geo Correlator } 
\STitle{The Bonn Correlator } 
\PublicationType{report} 
\PublicationName{IVS 2009 Annual Report}
\IvsComponentName{Bonn MPIfR/BKG Correlator}
%
\NumberofAuthors{4}
\AuthorName{1}{Simone Bernhart}
\AuthorName{2}{Alessandra Bertarini}
\AuthorName{3}{Arno M\"uskens}
\AuthorName{4}{Walter Alef}
% Contact_person: (Simone Bernhart)
% Contact_e-mail: (simone@mpifr-bonn.mpg.de)
\ContactAuthorReference{1}

\AuthorEmail{1}{simone@mpifr-bonn.mpg.de}
\AuthorTelephone{1}{+49-228-525-478}
\AuthorPersonalWebPage{1}{n.a.}

\AuthorEmail{2}{abertari@mpifr-bonn.mpg.de}
\AuthorTelephone{2}{+49-228-525-429}
\AuthorPersonalWebPage{2}{n.a.}

\AuthorEmail{3}{mueskens@mpifr-bonn.mpg.de}
\AuthorTelephone{3}{+49-228-525-264}
\AuthorPersonalWebPage{3}{http://igg.geod.uni-bonn.de}

\AuthorEmail{4}{walef@mpifr-bonn.mpg.de}
\AuthorTelephone{4}{+49-228-525-289}
\AuthorPersonalWebPage{4}{http://www.mpifr-bonn.mpg.de/staff/walef/}

\AuthorInstitutionReference{1}{1}
\AuthorInstitutionReference{2}{1}
\AuthorInstitutionReference{3}{1}
\AuthorInstitutionReference{4}{2}


\NumberofInstitutions{2}

\InstitutionName{1}{Institut f\"ur Geod\"asie  und Geoinformation der
Universit\"at Bonn}
\InstitutionPostAddress{1}{Nussallee 17, 53115 Bonn}
\InstitutionCountry{1}{Germany}
\InstitutionFax{1}{+49-228-732988}
\InstitutionWebPage{1}{http://igg.geod.uni-bonn.de}

\InstitutionName{2}{Max-Planck-Institute for Radioastronomy} 
\InstitutionPostAddress{2}{Auf dem H\"ugel 69, 53121 Bonn}
\InstitutionCountry{2}{Germany}
\InstitutionFax{2}{+49-228-525-225}
\InstitutionWebPage{2}{http://www.mpifr-bonn.mpg.de}


\begin{document}
\MakeTitle
%\thispagestyle{empty}

\begin{abstract}
The Bonn VLBI correlator is operated jointly by the MPIfR and the IGG in Bonn and the BKG in Frankfurt. The hardware Mark\,IV processor system was switched off in early December 2010 after a severe hardware failure and subsequently
geodetic correlation was moved over to the DiFX software correlator. Astronomical production correlation has been done exclusively with the DiFX correlator since autumn 2009.
\end{abstract}

\section{Introduction}
\label{sec:Intro}

\noindent The Bonn correlator is hosted at the Max-Planck-Institut f\"ur Radioastronomie (MPIfR)\footnote{\url{http://www.mpifr-bonn.mpg.de/div/vlbicor/}} Bonn, Germany. It is operated jointly by the MPIfR and the Bundesamt f\"ur Kartographie und Geod\"asie (BKG)\footnote{\url{http://www.bkg.bund.de/}} in cooperation with the
Institut f\"ur Geod\"asie und Geoinformation der Universit\"at Bonn (IGG)\footnote{\url{http://www.gib.uni-bonn.de/}}. It is a major correlator for geodetic observations and MPIfR's astronomical projects, for instance those involving millimetre wavelengths and astrometry. Astronomical experiments have been correlated with the DiFX software correlator since autumn 2009 while geodetic correlation was still ongoing on the Mark\,IV until December 3rd, 2010. The decommissioning of the hardware correlator, initially being scheduled for the end of 2010, was preponed due to a severe hardware failure which could not be repaired with an acceptable effort. 

\section{Present Correlator Capabilities}
\label{sec:Status}


\noindent In 2010, the Mark IV and DiFX correlator have been operated in parallel in Bonn. The DiFX correlator was mainly used for the correlation of astronomical experiments while the geodetic use was initially restricted to testing such as the feasibility of geodetic correlation and the implementation of the phase-cal signal extraction in the correlator software. The geodesists had to change over to the DiFX correlator earlier than planned due to an unexpected Mark\,IV hardware failure in early December.

\subsection{Mark\,IV correlator}

\noindent The Bonn hardware correlator was one of the four Mark IV VLBI data processors in the world. It has been operational from 2000 until early December 2010. It consisted of a standard Mark IV correlator rack, 7 Mark 5A units, 4 Mark 5B units, and one additional unit dedicated to eVLBI. The hardware Mark\,IV processor system and capabiliteis have not been changed since the last report and can be found here (http://ivscc.gsfc.nasa.gov/publications/ar2009/cobonn/).\\


\subsection{DiFX correlator}

\noindent DiFX (Distributed FX correlator) was developed at Swinburne University in Melbourne, Australia by Adam Deller (and collaborators), and adapted to the VLBA operational environment by Walter Brisken and NRAO staff. \\

\noindent Key parameters of the software correlator cluster are:
\begin{itemize}
 \item 60 nodes with 8 compute cores each (480 cores total)
 \item 4 TFlops (floating point operations) in the Linpack benchmark test
 \item Infiniband 20 Gb interconnect
 \item two times 1 Gb Ethernet interconnect
 \item two 20 TB raid systems
 \item “FXmanager” control computer which is the control node for the correlator
 \item “Frontend” control computer for users who use the cluster for other tasks than correlation
 \item Appliance control computer for installing and monitoring the cluster
 \item closed loop rack cooling
 \item Correlation  is done with the pre-release of DiFX version 2.0\footnote{astro-ph/-72141: DiFX: A software correlator for very long baseline interferometry using multi-processor computing environments} (DiFX-trunk)
 \item 14 Mark 5 units are connected to the software correlator. VLBI data can be played into the cluster from the Mark 5 recorders via 1 Gb Ethernet. If more than 14 playback units are required and in the case of e-VLBI, data are copied to the raid systems (presently 5) prior to correlation.
 \item All Mark 5s can play back all kinds of Mark 5 data (A/B/C).
 \item The fuse-based access to Mark 5 modules on all Mark 5s was changed to the native mode (implemented by Walter Brisken) making the Mark 5s part of the software cluster.
 \item The  Mark5A and Domino programs on the Mark 5s have been replace by NRAO's mk5daemon. It enables reading the module directories, i.e., the start and stop time of each scan; resetting the streamstor card; rebooting the Mark 5s; module conditioning.
\end{itemize}

\noindent The capabilities of the DiFX software correlator can be found at\\ http://www.mpifr-bonn.mpg.de/div/vlbicor/correlator\_e.html. Some important features are:
\begin{itemize}
 \item Phase-cal extraction of all tones in a sub-band simultaneously
 \item Interface to Mk\,IV data format which enables the use of geodetic analysis software and Haystack fringe fitting program.
\end{itemize}


\section{Staff}
\label{staff}

\noindent The people in the geodetic group at the Bonn correlator are\\
{\bf  Arno M\"uskens} - group leader, scheduling of T2, OHIG, EURO, INT3.  \\
{\bf  Alessandra Bertarini} - experiment setup and evaluation of correlated data, software correlator development. Digital baseband converter (DBBC) testing. Finished PhD at IGG Bonn in June 2010, subject of the thesis:
Effects on the geodetic-VLBI measurables due to polarization leakage in the receivers.\\
{\bf  Simone Bernhart} - eVLBI supervision and operations, experiment setup and evaluation of correlated
data, media shipping. Finished PhD at the MPIfR in March 2010, subject of the thesis: Flux Density and Kinematic Measurements of the IDV Source 0917+624.\\
{\bf  Laura La Porta} - experiment setup and evaluation of correlated data, DBBC testing.\\
{\bf  Rene B\"ockelmann} - setup and trial correlation of INT3.\\
{\bf  Fr\'ed\'eric Jaron} - phasecal extraction for software correlator, software support and web page maintenance.  \\
{\bf Six student operators} for the night shifts and the weekends.\\
\noindent The people in the astronomy group of MPIfR at the Bonn
correlator who support IVS correlation are\\
{\bf  Walter Alef} - head of the VLBI technical department, correlator
software maintenance and upgrades, computer system and cluster administration. Friend of the correlator.\\
{\bf  David Graham} - technical development, consultant, DBBC development and testing - retired in July 2010.\\
{\bf  Alan Roy} - deputy group leader, instrument scientist (water vapour radiometer, technical assistance, development of FPGA firmware for linear to circular polarization conversion, project manager for equipping APEX for millimetre VLBI).\\
{\bf  Helge Rottmann} - software correlator development and operation, cluster administration.\\
{\bf  Heinz Fuchs} - correlator operator, responsible for the correlator operator schedule, daily operations and media shipping.\\
{\bf  Hermann Sturm} - correlator operator, correlator support software, media shipping and web page development.\\
{\bf  Michael Wunderlich} - engineer, technical VLBI developments, (Mark IV correlator and) Mark 5 maintenance.\\
{\bf  Rolf M\"artens} - technician maintaining correlator hardware and Mark 5 playbacks.\\
{\bf  Gino Tuccari} - guest scientist from INAF, DBBC development, DBBC project leader.



\begin{figure}[h!]
\begin{center}
\epsfig{file=cobonn01.ps,width=0.8\textwidth}
\end{center}
\caption{Software correlator nodes.}
\label{fig:1}
\end{figure}


\section{Status}
\label{status}

\noindent {\bf Experiments}: In 2010 the Bonn group correlated 51 R1, seven EURO, six T2, eight OHIG, 44 INT3, and about 27 astronomical experiments (including tests and mm observations).\\
{\bf eVLBI}: e-transfers to Bonn are performed on a regular basis from Tsukuba, Ny-{\AA}lesund, Onsala, Mets\"ahovi, Wettzell, Kashima (including data of the antarctic Syowa station), Aira, Chichijima, Japanese VERA stations Ishigakijima (all from Tsukuba) and Mizusawa (from Mitaka), Medicina, Yebes and Hobart (since autumn 2010). Furthermore, successful tests have been performed between New Zealand and Bonn and between Fortaleza and Bonn. E-transfer reduces the time between observation and correlation since no shipment is required. The achieved data rates range from 100 Mb/s with Ny-{\AA}lesund (limited by radio link) to 600 Mb/s with peaks up to 800 Mb/s (with Mets\"ahovi). The transfers are done using the UDP-based Tsunami protocol. The total disk space available for eVLBI data storage at the correlator is currently about 46 Tbyte. A webpage has been developed (http://www.mpifr-bonn.mpg.de/cgi-bin/showtransfers.cgi) which shows currently active (Tsunami) e-transfers. This is not intended as a scheduling tool for upcoming e-transfers but should rather help to coordinate transfer times and rates on a first come-first served base.  \\ 
{\bf Hardware correlator}: 6 Mark5A and 5 Mark5B units were available for the hardware correlator. One additional Mark 5 unit was dedicated to evlbi. The Mark IV correlator was shut down at the beginning of December due to a broken control board. A repair would have taken longer than the initially scheduled replacement by the DiFX correlator at the end of 2010.\\ 
{\bf DiFX software correlator}: The implementation of the phase-cal signal extraction into the DiFX correlator software was finished and verified for geodetic experiments; the first R1 was correlated without phase-cal extraction in May. Since September, further R1s and one EURO experiment had been correlated on both the Mark\,IV and DiFX correlator and the results were compared concerning their compatibility. The abrupt changeover to the DiFX correlator in early December, however, made it necessary to iron out a few bugs in the conversion software from DiFX to Mark\,IV format causing delays in the submission of the experiment databases. Nevertheless, the use of geodetic analysis software and Haystack fringe fitting program is now working which enables the delivery of the databases back on schedule. For the DiFX correlator, no station units are required anymore. Hence, they have been removed from the Mark 5 racks and the units are only used for data playback.\\ 
{\bf DBBC}: The Bonn group is involved in the development of a DBBC for the European VLBI Network (EVN) and geodesy. The DBBC is designed as a full replacement for the existing analogue BBCs. The following stations already bought one or more DBBCs: Wettzell (3), Latvia, Yebes, AuScope (Australia) and the functionality of these DBBCs is currently tested. Hobart12, one of the AuScope stations, already uses it in their regular observations.


\section{Outlook for 2011}
\label{outlook}

\noindent {\bf DiFX Correlator}: There are still some minor bugs in the conversion software from DiFX to Mark IV format which need to be fixed. Some minor issues in the export software are currently tried to be ironed out as well. Furthermore a graphical user interface will be installed on the DiFX control computer in the near future to simplify the use of the software correlator.\\
{\bf eVLBI}: Stream correlation using eVLBI transfer will continue and eVLBI tests with other antennas are planned/ongoing. An additional 70 TB data raid will be installed at the cluster middle of February to increase the storage capacity for correlated geodetic data and future e-transfers, especially with regard to the envisaged higher observing rate of 512 Mbps in the course of VLBI2010. In order to meet the requirements of the higher observing rate, there are still plans to upgrade our internet connection from 1Gbps to 2Gbps. However, the funding and technical implementation prove to be more than difficult.\\
{\bf DBBC}: There is ongoing testing among several EVN stations in order to deploy a fully functional DBBC and replace the analogue BBCs in the field.



\end{document}

