Operating TMS (updated 19-Jan-2025)

Contents

The software that controls TMS — requesting measurements and using these to update the position of the primary mirror — cannot run simultaneously with the FPIA software used to focus and collimate; and so, since 2023B, a software service running in the background will stop the TMS loop whenever the observer starts FPIA (dofpia or dohybrid) and, upon successful completion of FPIA, it will set a new TMS reference and resume the TMS loop.

In this mode of operation, FPIA should no longer be run from the IDL window, but from a terminal window at the Linux shell prompt, using a slightly modified command syntax. The old syntax used commas to separate keywords, e.g. IDL> dofpia, /x2, whereas the new syntax uses the double-dash to indicate keyword options, e.g. obs1> dofpia --x2

This page should contain all that is needed for an observer to run TMS when observing with the LBCs.

(This page was last updated 19-Jan-2025 to clarify the procedure and to highlight the times when the observer needs to wait for either the reference-taking and or the mirror motions commanded by TMS to complete, as summarized in the Notes section. It is still under construction, but sufficiently complete to post.)

Quick Review

1. Turn on the TMS lasers and send an email to telescopework@lbto.org to let everyone know the lasers are ON. Instructions for turning the lasers ON and OFF can be found at this link. It is recommended to turn on the lasers 15-20 minutes before the start of LBC observations to give them time to warm up and to troubleshoot any potential issues. For example, turn them on as soon the operator has started to reconfigure for the LBCs.
2. Start the TMS loop (using a previous reference or after the initial focus and collimation of the night):
   1. Using a previous reference: If you want to use a previous reference, then you can start the TMS loop before you focus and collimate, even before the enclosure is opened. By default, the last reference will be used. See Using the last good reference as a starting point, for instructions on selecting a previous reference other than the last one.
      1. Start the TMS loop by typing in a terminal window (it can be the same one as used in the previous step):
         1. tms_loop  for binocular observations (the default);
         2. tms_loop -s 0 (SX/LBCB); or
         3. tms_loop -s 1 (DX/LBCR)
      2. Start the “watcher” scripts: These monitor TMS corrections and new TMS references.
         1. Open 4 small terminal windows. It is helpful to arrange them in a square grid and keep all of them visible.
         2. In the upper left terminal window, type watch_tms_left, and in the upper right one type watch_tms right.
         3. In the lower left terminal windows, type watch_ref_left, and in the lower right one type watch_ref_right.
      3. Start LBTplot by typing LBTplot in a terminal window. The SXTMS and DXTMS tabs will show TMS corrections. (LBTplot does not currently run from obs1-5, 17-Sep-2024).
      4. Slew to the copointing target for your first field. Stop the OB before any images are taken.
         1. For very low elevation targets, it is better to collimate at higher elevation where the seeing is better (~65-75 deg, though not > 80 deg because, here, rotational trailing may be an issue).
         2. When on-target, in a terminal window, type dofpia (or  dofpia --x2) to focus and collimate. See Observing Procedure –> Collimating for all of the options which dofpia and dohybrid can take.
      5. The TMS/FPIA background service will automatically take a reference upon successful completion of dofpia. Wait for the reference-taking to compete before playing the OB again. Skip to 3I.
   2. Without using a previous reference: If you don’t know or are unsure about using a previous reference, then run FPIA and manually set a new reference before starting the TMS loop, as described here:
      1. Slew to the copointing target for your first field. Stop the OB before any images are taken.
         1. For very low elevation targets, it is better to collimate at higher elevation where the seeing is better (~65-75 deg, though not > 80 deg because, here, rotational trailing may be an issue).
      2. When on-target, in a terminal window, type dohybrid (or  dohybrid --x2) to focus and collimate. See Observing Procedure –> Collimating for all of the options which dofpia and dohybrid can take.
      3. After dohybrid has converged, manually take a TMS reference. In a terminal window, type:
         1.  lbc_set_tms_offset -s 2 to set a reference on both sides (i.e. for both LBCB and LBCR);
         2. lbc_set_tms_offset -s 0 to set a reference on the SX (LBCB) side only; or
         3. lbc_set_tms_offset -s 1 to set a reference on the DX (LBCR) side only.
      4. Wait for the reference to complete, and then start the TMS loop by typing in a terminal window (it can be the same one as used in the previous step):
         1. tms_loop  for binocular observations (the default);
         2. tms_loop -s 0 (SX/LBCB); or
         3. tms_loop -s 1 (DX/LBCR)
      5. Start the “watcher” scripts: These monitor TMS corrections and new TMS references.
         1. Open 4 small terminal windows. It is helpful to arrange them in a square grid and keep all of them visible.
         2. In the upper left terminal window, type watch_tms_left, and in the upper right one type watch_tms right.
         3. In the lower left terminal windows, type watch_ref_left, and in the lower right one type watch_ref_right.
      6. Start LBTplot by typing LBTplot in a terminal window. The SXTMS and DXTMS tabs will show TMS corrections. (LBTplot does not currently run from obs1-5, 17-Sep-2024).
3. With the TMS loop running, slew to your copointing field, but stop the OB and wait for TMS corrections to be sent and for the primaries to stop moving before playing the OB again.  When a TMS correction is sent, the watch_tms_left and watch_tms_right will update with a time-stamped entry showing the target’s elevation and status = “leftactive” or “rightactive”. When the primary mirror status on the IIFGUI changes from a yellow to black background, the primaries have stopped moving.
   1. This time allow the OB to run to completion to take the pair of images. In your IRAF xgterm window, run lbcrangebal (LBTtools.observe.lbcrangebal) to correct pointing & copointing. Note that with TMS running, the pointing and copointing corrections may be larger than normal.
4. Slew to your target by playing the science OB, but then stop the OB and wait for TMS corrections to be sent and for the primaries to stop moving before playing the OB again. Once the mirrors have stopped moving, you can start the science OB!
5. For each new target, execute steps 3 and 4 to correct pointing and copointing and observe the target.
   1. Assess image quality and if necessary run dofpia to focus and collimate. Image quality can be assessed in at least two ways:
      1. Refresh the link to this plot: https://people.lbto.org/~cveillet/dms/LBC_Current_Plot.png of the average FWHM and ellipticity for the LBCB and LBCR images, together with the DIMM seeing corrected to the telescope elevation.
      2. Use imexam “r” to measure the FWHM on the images displayed by RB_Science.
   2. If you are tracking a single target through high elevation (>75-80 deg) and the pointing and copointing were only corrected at lower elevation, it is recommended to correct pointing and copointing once again before reaching ~75-80 degrees.
   3. If the target is near the previous one, correcting pointing and copointing may not be necessary.
6. After finishing the LBC observations:
   1. Stop the TMS loop (control-C in the terminal window where you launched tms_loop).
   2. Turn the TMS lasers OFF and send an email to telescopework@lbto.org to let everyone know that the lasers are now off.

Important Notes:

1. Wait until TMS is finished taking a reference before playing a new OB.
   1. Look for new time-stamped entries in the watch_ref_left and watch_ref_right windows.
2. After you slew to a new field, always wait for a new TMS update before starting to take an exposure.
   
   1. Look for new time-stamped entries at the target elevation and with status “leftactive” or “rightactive” in the watch_tms_left and watch_tms_right windows. These indicate that a TMS correction has been sent to the primary mirrors, but the motion may still be ongoing. To determine whether the primaries have finished moving, look for the primary mirror status on the IIFGUI; when the mirrors are moving, the background is yellow, but when they have stopped moving, the background will turn black.

Glossary of Commands

The scripts needed to run TMS and watch for updates are in the path for all partner accounts on these workstations, so to start the program the user need only type the program name from any directory. For reference, the full path to the TMS scripts is: /lbt/lbto/supportscripts/MetrologySupport

• tms_loop   — runs the TMS correction loop.
  • tms_loop for binocular operation (the default)
  • tms_loop -s 0 for operation only with LBCB on SX
  • tms_loop -s 1 for operation only with LBCR on DX
• lbc_set_tms_offset   — sets a new TMS reference manually
  • lbc_set_tms_offset -s 2 to set a new reference on both sides
  • lbc_set_tms_offset -s 0 to set a new reference on SX only
  • lbc_set_tms_offset -s 1 to set a new reference on DX only
• watch_tms_left, watch_tms_right — monitors TMS corrections
  • These show the TMS corrections that are being sent (labelled active) as well as reasons why any TMS corrections that may have been calculated are not being sent (mirror or telescope moving, shutter open).
• watch_ref_left, watch_ref_right — monitors TMS references
• dataloop — testing/reporting info on raw etalon length (for troubleshooting – will be added to that section)
• tmsplot — plot TMS results

Using the last good reference as a starting point

Be aware that an old reference will not work if the channels used for it do not match the currently available channels. This may result:

1. if VXWorks has gone down, causing a mirror panic (a special case of mirror panic).
2. If work near the primary has accidentally disturbed one or more collimators.
3. from damage to any of the fibers

By default, the tms_loop uses the last reference taken. If you know it is reasonable (taken in the last night or during the ongoing observing run), then using it can help to reduce the time-to-collimate at the beginning of the night or to improve the focus and collimation for any stars that might be in the start-of-night flat fields (over what they might have without it).

1. References are stored in /lbt/data/share/tms/refs/<UTdate> — the first one will have a name like ***-<UTdate>.dat and subsequent ones will be numbered as ***-<UTdate>-#.dat, with the most recent reference having the largest index number.
2. tms_loop with no arguments will use the most recent previous references on SX and DX.
3. To select an earlier reference, issue the tms_loop command with the argument --ref as indicated in the examples below:

1. 1. tms_loop.py -s 1 --ref 'dx:20240529-3' will run the active loop only on DX and use the 4th DX reference taken on UT 20240529
   2. tms_loop.py --ref 'dx:20240529' will run the active loop on both sides but use the latest reference on SX and the first reference from 20240529 on DX.

Troubleshooting

1. If there are problems taking the reference:
   1. Check for missing channels. TMS requires at least 6 channels, at least 1 and no more than 4 on each of the 3 retroreflectors that are mounted on the LBC.
      1. Missing channels may be due to a misalignment. This has been known to occur after power bumps when the mirror goes down. If this is the cause, then the collimators will need to be realigned during the day.
      2. Missing channels may also result from system malfunction. Recent cases like this have been resolved by power cycling the TMS cabinet.
2. If no corrections are being sent, check for missing channels in the screen output from tms_loop.py.
   1. But be aware that, TMS will not send corrections when:
      1. the LBCs are exposing;
      2. telescope is slewing;
      3. mirrors are moving;
      4. dofpia or dohybrid is being run.
3.  The first pointing/copointing corrections after setting the collimation from an old reference may be large (Large enough to violate or warn about violating the copointing limit). If in doubt, retake the copointing pair (when this has happened, the repeated images have shown that the pointing and copointing corrections were made).
4. The tmsfpia synchronization service that runs on the dms machine should restart automatically in the event of a crash, but sometimes it does not.
   1. To check that this service is running, type (from <partner>@robsN or <partner>@obsN):
      1. nc dms.mountain.lbto.org 39999
         1. If it reports “Ncat: Connection refused.”, then the service will need to be reset.
         2. If it hangs, then the service is up and you can enter Cntl-D to return to the prompt, or type “state query” for additional information and then Cntl-D.
   2. To restart the tmsfpia sync service, the support staff will need to:
      1. login to the dms machine: ssh swg@dms.mountain.lbto.org
      2. at the terminal prompt, type: sudo systemctl restart lbto_tmsfpiasync
5. Sometimes (rarely) the tmsfpia sychronization service only needs to be reset.
   1. To reset the tmsfpia sync service:
      1. type nc dms.mountain.lbto.org 39999
      2. type __reset__  (these are double-underscores) and press the enter key
      3. it should echo with “done”

For more details

More detailed information about TMS operation can be found at this wiki page and links therein. This page provides the syntax for running the TMS loop in engineering modes.

PSF GUIs (with TMS corrections labelled)