Skip to content

Commit fea3547

Browse files
DM-56308: updated DP2 NB 306 1 and new 306 2 (extragal transients) (#220)
* updated 306 1 and new 306 2 * incorporate SF comments * incorporate SF comments
1 parent 290d91b commit fea3547

2 files changed

Lines changed: 904 additions & 24 deletions

File tree

‎DP2/300_Science_demos/306_Extragalactic_transients/306_1_Transient_light_curves.ipynb‎

Lines changed: 112 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@
2222
"Data Release: [Data Preview 2](https://dp2.lsst.io/)\\\n",
2323
"Container Size: Large\\\n",
2424
"LSST Science Pipelines version: r30.0.11\\\n",
25-
"Last verified to run: 2026-08-18\\\n",
25+
"Last verified to run: 2026-10-16\\\n",
2626
"Repository: [github.com/lsst/tutorial-notebooks](https://github.com/lsst/tutorial-notebooks)\\\n",
2727
"DOI: [10.11578/rubin/dc.20250909.20](https://doi.org/10.11578/rubin/dc.20250909.20)"
2828
]
@@ -36,7 +36,7 @@
3636
"\n",
3737
"**LSST data products:** `ForcedSourceOnDiaObject`, `DiaObject`, `Visit`\n",
3838
"\n",
39-
"**Packages:** `matplotlib`, `numpy`, `lsst.rsp`, `lsst.utils.plotting`\n",
39+
"**Packages:** `lsst.rsp`\n",
4040
"\n",
4141
"**Credit:**\n",
4242
"Originally developed by the Rubin Community Science team with feedback from Eric Bellm.\n",
@@ -58,11 +58,10 @@
5858
"\n",
5959
"This notebook demonstrates how to obtain and plot a difference-image light curve of an extragalactic transient.\n",
6060
"\n",
61-
"**Recommendations for extragalactic transient light curves.**\n",
62-
"\n",
63-
"For extragalactic transients, it is recommended to create light curves using forced difference-image photometry.\n",
64-
"Using the difference-image photometry ensures that any underlying host galaxy light does not contaminate the flux measurement, as can happen if science-image (or direct- or visit-image) photometry is used.\n",
65-
"Using the forced photometry ensures that a data point exists for every observation that overlapped the transient's coordinates, not just those in which it was detected at $>5\\sigma$ in the difference image.\n",
61+
"**For light curves of extragalactic transients, it is recommended to use forced flux measured on the difference image.**\n",
62+
"* Using the **difference-image** photometry ensures that any underlying host galaxy light does not contaminate the flux measurement, as can happen if science-image (or direct- or visit-image) photometry is used.\n",
63+
"* Using the **forced** photometry ensures that a data point exists for every observation that overlapped the transient's coordinates, not just those in which it was detected at $>5\\sigma$ in the difference image.\n",
64+
"* Using the **fluxes** is important because difference-image fluxes can be negative and thus should not be converted to magnitudess.\n",
6665
"\n",
6766
"The forced difference-image photometry is stored in the `ForcedSourceOnDiaObject` table, and the relevant columns are:\n",
6867
"* `diaObjectId`: Unique DiaObject identifier\n",
@@ -72,7 +71,11 @@
7271
"\n",
7372
"Observation dates (MJDs) are stored in the `Visit` table, in column `expMidptMJD` (exposure midpoint MJD), and are obtained with a table join.\n",
7473
"\n",
75-
"Difference-image fluxes can be negative and thus should not be converted to magnitudes, as without care this can lead to missing data points (especially observations with non-detections, which can be scientifically very useful for transients).\n",
74+
"**Be aware that some template images contain transient flux (\"template contamination\").**\n",
75+
"In the future, the template images will be created from the previous year's data, but for DP2 this is not possible.\n",
76+
"The third (33%) of the visit images with the best seeing were combined to create the template images, and in some cases flux from transient objects \"contaminates\" the template image (e.g., when many images of a given sky region were obtained in a short amount of time). \n",
77+
"In these cases, transients will appear to have negative difference-image flux in some epochs.\n",
78+
"This tutorial will demonstrate how to identify such cases.\n",
7679
"\n",
7780
"**Related tutorials:** There are 200-level tutorials on `difference_images` as well as the `DiaSource`, `DiaObject` and `ForcedSourceOnDiaObject` catalogs."
7881
]
@@ -95,7 +98,6 @@
9598
"`matplotlib`, a comprehensive library for data visualization\n",
9699
"(<a href=\"https://matplotlib.org/\">matplotlib.org</a>; \n",
97100
"<a href=\"https://matplotlib.org/stable/gallery/index.html\">matplotlib gallery</a>).\n",
98-
"\n",
99101
"From the `lsst` package, import modules for accessing the Table Access Protocol (TAP) service and image display functions from the LSST Science Pipelines (<a href=\"https://pipelines.lsst.io/\">pipelines.lsst.io</a>)."
100102
]
101103
},
@@ -110,7 +112,8 @@
110112
"import numpy as np\n",
111113
"from lsst.rsp import RSPDiscovery\n",
112114
"from lsst.utils.plotting import (get_multiband_plot_colors,\n",
113-
" get_multiband_plot_symbols,)"
115+
" get_multiband_plot_symbols,\n",
116+
" set_rubin_plotstyle)"
114117
]
115118
},
116119
{
@@ -155,6 +158,7 @@
155158
"metadata": {},
156159
"outputs": [],
157160
"source": [
161+
"set_rubin_plotstyle()\n",
158162
"filter_colors = get_multiband_plot_colors()\n",
159163
"filter_names = filter_colors.keys()\n",
160164
"filter_symbols = get_multiband_plot_symbols()"
@@ -167,20 +171,33 @@
167171
"source": [
168172
"## 2. Retrieve the light curve data\n",
169173
"\n",
170-
"Light curve data is retrieved from the `ForcedSourceOnDiaObject` by the associated `diaObjectId`, not by coordinates.\n",
171-
"\n",
172-
"If the `diaObjectId` of the time-domain object for which the light curve is desired is already known, skip to Section 2.2.\n",
174+
"Light curve data - forced difference-image fluxes - are retrieved from the `ForcedSourceOnDiaObject` table, using the `diaObjectId` column.\n",
175+
"The `ForcedSourceOnDiaObject` table cannot be queried by coordinate; all spatial queries for time-domain objects must be done on the `DiaObject` table.\n",
173176
"\n",
177+
"* Section 2.1 demonstrates the two step process to retrieve first the `diaObjectId` for the extragalactic transient using a spatial query on the `DiaObject` table.\n",
178+
"* Section 2.2 demonstrates how to then retrieve the light curve data from the `ForcedSourceOnDiaObject` table using a query on the `diaObjectId` column.\n",
179+
"* Section 2.3 demonstrates how to combine these two steps into a single query using table joins."
180+
]
181+
},
182+
{
183+
"cell_type": "markdown",
184+
"id": "f49ec673-dd38-49a3-9c90-26ef1bf5c537",
185+
"metadata": {},
186+
"source": [
174187
"### 2.1. Spatial query for the diaObject\n",
175188
"\n",
176-
"**Spatial queries require a table join:** \n",
189+
"**Spatial queries should be done on `DiaObject`, not on the source tables.**\n",
177190
"Spatial queries for time-domain objects are done on the `DiaObject` table, *not* on the tables containing the measured fluxes at the same location in every image.\n",
178-
"Spatial queries on `ForcedSourceOnDiaObject` (or `DiaSource`) are inefficient and for this reason, coordinates are not included in the `ForcedSourceOnDiaObject` table.\n",
191+
"Spatial queries on `ForcedSourceOnDiaObject` (or `DiaSource`) are inefficient.\n",
192+
"Coordinates are not included in the `ForcedSourceOnDiaObject` table for this reason.\n",
193+
"\n",
194+
"**Set the cross-match radius a bit high (2\" instead of 1\") for this demonstration.**\n",
195+
"Difference-image detections are associated within a 1\" radius, and typically that is a fine search radius to use for point-source cross-match (and is used in Section 2.3).\n",
196+
"However in the query below a 2\" radius is used to mimic a situation in which, e.g., the transient was detected in a lower-resolution survey, in order to demonstrate how to deal with multiple potential cross-match results.\n",
179197
"\n",
180-
"**Cross-match radius:** \n",
181-
"Although difference-image detections are associated within a 1\" radius, and typically that is a fine search radius to use for point-source cross-match, a 2\" radius is used for this example. It mimics a situation in which, e.g., the transient was detected in a lower-resolution survey, and demonstrates how to deal with multiple potential cross-matches. For scenarios in which cross-match for a *set* of coordinates is desired, instead of a single coordinate, see the tutorials on cross-match to user-uploaded tables. \n",
198+
"For scenarios in which cross-match for a *set* of coordinates is desired, instead of a single coordinate, see the tutorials on cross-match to user-uploaded tables. \n",
182199
"\n",
183-
"Define the coordinates of a known transient and the search radius. Query the `DiaObject` table and retrieve the `diaObjectId`, the number of detections (`nDiaSources`), and the offset distance between all matches and the search coordinates. Retrieve the results as an Astropy table."
200+
"Define the coordinates of a known transient ([SN2025mbp](https://www.wis-tns.org/object/2025mbp)) and the search radius. Query the `DiaObject` table and retrieve the `diaObjectId`, the number of detections (`nDiaSources`), and the offset distance between all matches and the search coordinates. Note that this is not a nearest-neighbor search and that multiple matches within the search radius may be returned. Retrieve the results as an Astropy table."
184201
]
185202
},
186203
{
@@ -192,7 +209,9 @@
192209
"source": [
193210
"ra_targ = 151.592579\n",
194211
"dec_targ = 1.158333\n",
212+
"\n",
195213
"search_radius = 2.0/3600.0\n",
214+
"\n",
196215
"query = \"\"\"SELECT diaObjectId, nDiaSources,\n",
197216
" DISTANCE(POINT('ICRS', ra, dec), POINT('ICRS', {}, {}))\n",
198217
" FROM dp2.DiaObject\n",
@@ -241,8 +260,9 @@
241260
"source": [
242261
"### 2.2. Query for forced photometry\n",
243262
"\n",
244-
"Create a query to retrieve the flux, band, and MJD of the diaObject using a table join on `ForcedSourceOnDiaObject` and `Visit`.\n",
245-
"Retrieve the results as an Astropy table."
263+
"Create a query to retrieve the difference-image forced flux, its error, and the band (filter) from the `ForcedSourceOnDiaObject` table, for all rows associated with the `diaObjectId`.\n",
264+
"Use a table join to retrieve the MJD of the observation from the `Visit` table.\n",
265+
"Retrieve the results as an Astropy table, then print the table length (number of measurements)."
246266
]
247267
},
248268
{
@@ -253,6 +273,7 @@
253273
"outputs": [],
254274
"source": [
255275
"use_diaObjectId = 786352299365629972\n",
276+
"\n",
256277
"query = \"\"\"SELECT fsodo.psfDiffFlux, fsodo.psfDiffFluxErr,\n",
257278
" fsodo.band, v.expMidptMJD\n",
258279
" FROM dp2.ForcedSourceOnDiaObject AS fsodo\n",
@@ -266,6 +287,7 @@
266287
" job.raise_if_error()\n",
267288
"assert job.phase == 'COMPLETED'\n",
268289
"flux_table = job.fetch_result().to_table()\n",
290+
"\n",
269291
"print(len(flux_table))"
270292
]
271293
},
@@ -287,6 +309,64 @@
287309
"# flux_table"
288310
]
289311
},
312+
{
313+
"cell_type": "markdown",
314+
"id": "b7d44df7-69bd-4063-948d-67d082700577",
315+
"metadata": {},
316+
"source": [
317+
"Delete the first two query results, as they will be repeated as a single three-table join in the next section."
318+
]
319+
},
320+
{
321+
"cell_type": "code",
322+
"execution_count": null,
323+
"id": "694f0280-604d-4ff2-b2c6-426c854fbe51",
324+
"metadata": {},
325+
"outputs": [],
326+
"source": [
327+
"del diao_table, flux_table"
328+
]
329+
},
330+
{
331+
"cell_type": "markdown",
332+
"id": "c3039cb5-ffff-4a86-bc0d-228c28a1c854",
333+
"metadata": {},
334+
"source": [
335+
"### 2.3. Combine steps into a triple-table query\n",
336+
"\n",
337+
"**Lower the cross-match radius to 1\"** so that only one `diaObject` is returned from the spatial query.\n",
338+
"\n",
339+
"Combine the two queries above into a single query with a triple-table join. Execute the query job and return the results as an Astropy table, and print the number of rows."
340+
]
341+
},
342+
{
343+
"cell_type": "code",
344+
"execution_count": null,
345+
"id": "39efd36a-03a5-4bed-9a00-e1ad623db1ba",
346+
"metadata": {},
347+
"outputs": [],
348+
"source": [
349+
"search_radius = 1.0/3600.0\n",
350+
"\n",
351+
"query = \"\"\"SELECT diao.diaObjectId, fsodo.psfDiffFlux, fsodo.psfDiffFluxErr,\n",
352+
" fsodo.band, v.expMidptMJD\n",
353+
" FROM dp2.DiaObject AS diao\n",
354+
" JOIN dp2.ForcedSourceOnDiaObject AS fsodo ON fsodo.diaObjectId = diao.diaObjectId\n",
355+
" JOIN dp2.Visit AS v ON v.visit = fsodo.visit\n",
356+
" WHERE CONTAINS(POINT('ICRS', diao.ra, diao.dec), CIRCLE('ICRS', {}, {}, {})) = 1\n",
357+
" \"\"\".format(ra_targ, dec_targ, search_radius)\n",
358+
"job = tap_service.submit_job(query)\n",
359+
"job.run()\n",
360+
"job.wait(phases=['COMPLETED', 'ERROR'])\n",
361+
"print('Job phase is', job.phase)\n",
362+
"if job.phase == 'ERROR':\n",
363+
" job.raise_if_error()\n",
364+
"assert job.phase == 'COMPLETED'\n",
365+
"flux_table = job.fetch_result().to_table()\n",
366+
"\n",
367+
"print(len(flux_table))"
368+
]
369+
},
290370
{
291371
"cell_type": "markdown",
292372
"id": "89907542-962f-4dbd-9632-d9f1788c846d",
@@ -369,9 +449,11 @@
369449
"source": [
370450
"### 3.1. Take care with magnitudes\n",
371451
"\n",
372-
"Do not convert magnitudes to difference-image fluxes, as any negative fluxes will be lost. Negative difference-image fluxes are not erroneous -- they simply mean the time-varying object was brighter in the template than in the science image.\n",
452+
"Do not convert magnitudes to difference-image fluxes, as any negative fluxes will be lost. Negative difference-image fluxes are not erroneous: they indicate that the time-varying object was brighter in the template than in the science image.\n",
373453
"\n",
374-
"Re-plot the light curve in Figure 2, but with magnitudes, to show the error message and that some data points go missing. There will be a pink \"RuntimeWarning\" to represent negative values passed to `np.log10`."
454+
"Re-plot the light curve in Figure 2, but with magnitudes, to show the error message and that some data points go missing. \n",
455+
"\n",
456+
">**Warning:** There will be a pink \"RuntimeWarning\" to indicate that negative values were passed to `np.log10`."
375457
]
376458
},
377459
{
@@ -422,6 +504,7 @@
422504
"outputs": [],
423505
"source": [
424506
"use_diaObjectId = 786352299365629972\n",
507+
"\n",
425508
"query = \"\"\"SELECT psfFlux, psfFluxErr, band, midpointMjdTai\n",
426509
" FROM dp2.DiaSource AS dias\n",
427510
" WHERE diaObjectId = {}\"\"\".format(use_diaObjectId)\n",
@@ -433,6 +516,7 @@
433516
" job.raise_if_error()\n",
434517
"assert job.phase == 'COMPLETED'\n",
435518
"dias_table = job.fetch_result().to_table()\n",
519+
"\n",
436520
"print(len(dias_table))"
437521
]
438522
},
@@ -483,13 +567,17 @@
483567
"id": "014f78e1-1913-4a8f-8f0e-f17e9a10e6a6",
484568
"metadata": {},
485569
"source": [
486-
"> **Figure 4:** Notice that data points for some epochs, especially those in which the difference-image flux was close to 0, are missing from the bottom plot. For forced, difference-image photometry, a flux of zero is still an interesting and useful data point. But these points are missed if only the detections in the `DiaSource` table are used instead of all forced measurements in `ForcedSourceOnDiaObject`."
570+
"> **Figure 4:** The top panel is the same as Figure 2 (forced PSF difference-image fluxes from the `ForceSourceOnDiaObject` table), but the bottom panel is the detected difference-image flux from the `DiaSource` table.\n",
571+
"\n",
572+
"**Missing epochs:** Notice that data points for some epochs, especially those in which the difference-image flux was close to 0, are missing from the bottom plot. For forced, difference-image photometry, a flux of zero is still an interesting and useful data point - but these points are missed if only the detections in the `DiaSource` table are used instead of all forced measurements in `ForcedSourceOnDiaObject`.\n",
573+
"\n",
574+
"**Template contamination:** Notice also that the first epochs have difference-image fluxes less than zero, which is indicative of some template contamination in those filters at the location of this supernova in Data Preview 2. For future data releases, when template coadd images are built from past years of images, template contamination will be less of an issue (except for very long-duration transients)."
487575
]
488576
},
489577
{
490578
"cell_type": "code",
491579
"execution_count": null,
492-
"id": "5342519a-f6ce-4890-b2ad-66088735fcd5",
580+
"id": "346764ae-c246-48cb-9a81-a3da86209665",
493581
"metadata": {},
494582
"outputs": [],
495583
"source": []

‎DP2/300_Science_demos/306_Extragalactic_transients/306_2_Cross_match_queries.ipynb‎

Lines changed: 792 additions & 0 deletions
Large diffs are not rendered by default.

0 commit comments

Comments
 (0)