Doing TBB and pulsar observations from lofarx

1) Prepare the shell on lofarx with a running ssh-agent:

• log in on lofarx with "ssh observer@lofarx" from a machine in the mpifr LDAP network (e.g. your desktop or portal or so.)
  • if it asks for a password, don't bother, it won't work. You can try running "kinit" on you local machine before doing the ssh.
• ssh-agent /bin/tcsh -l
• ssh-add
  • give passphrase

2) Boot up the station:

• log into the LCU with "ssh de601c"
• Check the destination of the beam-formed (pulsar) data:
  • grep lofar /opt/lofar/etc/RSPDriver.conf

  • The result should look like this for the lofarAN computers:

    RSPDriver.LANE_00_DSTMAC  = 00:30:48:34:74:5f   # lofarA1  
    RSPDriver.LANE_01_DSTMAC  = 00:30:48:34:74:73   # lofarA2
    RSPDriver.LANE_02_DSTMAC  = 00:30:48:34:74:69   # lofarA3
    RSPDriver.LANE_03_DSTMAC  = 00:30:48:bb:90:2f   # lofarA

    and like this for the lofarBN computers:

    RSPDriver.LANE_00_DSTMAC  = 00:19:99:ba:42:cb   # lofarB1  
    RSPDriver.LANE_01_DSTMAC  = 00:19:99:e2:21:46   # lofarB2
    RSPDriver.LANE_02_DSTMAC  = 00:19:99:e2:21:ba   # lofarB3
    RSPDriver.LANE_03_DSTMAC  = 00:19:99:e2:20:ce   # lofarB4 

  • (The destination for the TBB data is chosen later byt thedump-script, and does not require change of the swlevel)
• swlevel 2
  • RSPDriver and TBBDriver need to be running
• tail -f /log/RSPDriver.log
  • wait till no more lines: "port ??? has not yet completed sync or had errors, trying to continue..." show up for several seconds
  • If anything strange appears, write it down in the observing logbook and tell the station team: lofar-ops@mpifr-bonn.mpg.de
  • end it with <CTRL-C>
• tbbctl --version
  • all 12 TBB-boards should show up with their firmware and software versions, if one is "board not active", then please contact the station team for a "48V reset"
• swlevel 3
  • In addition to RSPDriver and TBBDriver also BeamServer needs to be running
• beamctl --antennaset=HBA_JOINED --rcus=0:191 --rcumode=5 --beamlets=0:243 --subbands=100:343 --anadir=0,1.5708,J2000 --digdir=0,1.5708,J2000&
• beamctl --antennaset=LBA_OUTER --rcus=0:191 --rcumode=3 --beamlets=0:243 --subbands=100:343 --digdir=0,1.5708,J2000&
  • This should reach "All pointings sent and accepted" pretty fast.
  • hit <enter> to get a clean commandline
  • Wait several seconds for all tiles to start up
• rspctl --rcu
  • all 192 RCUs should be "ON" and in "mode:5"
  • if some are "OFF", wait a few seconds and try again
  • if some don't become "ON" then go bach to swlevel 1 and try again
• Check if the beam-formed data arrives at the correct computers:
  • in another window, log into the data recording-computer and run:
  • sudo tcpdump -ni eth1
  • you should see lots of packets arriving
• rspctl --stati --int=3
  • check if the spectrum of all antennas look O.K.
• killpointing
  • hit <enter> to get the confirmation that beamctl terminated

3) Observe

• Go back to lofarx
• Run one of the scripts:
  • tbb-psr-observe.py                       (only TBBs)
  • tbb-psr-sametime-observe.py
  • tbb-psr-HBAsingle-observe.py
• The scripts should manage everything you need for a TBB dump with interleaved pulsar observations.  Call it with "-h" for help.

4) Finish up:

• check with "ps -fluobserver" if there are lingering jobs on the lofarN machines. If there are, please kill them.
• shut down the hardware of the station with "swlevel 0"

Comments:

1. If at any time RSPDriver, TBBDriver, or BeamServer crash ("swlevel" show them as "DOWN") you need to go back to swlevel 1 and boot the station up again. (And note the fact in the logbook.)
2. If nothing fails there is no need to change the swlevel on the station when observing multiple pulsars. Just leave it in "swlevel 3"