{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "This tutorial covers some of the basic functionality of the sigpyproc package. For a guide on how to extend the\n", "package, see the" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Getting started" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For test puproses, few small size filterbank files are included in the `/tests/data/` directory of the `sigpyproc` package." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Loading data into sigpyproc" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Lets start by loading our filterbank file into sigpyproc. To do this, we require the :class:`~sigpyproc.readers.FilReader` class from the :automodule:`sigpyproc.readers` module." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "from rich.pretty import Pretty\n", "from sigpyproc.readers import FilReader" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "myFil = FilReader(\"../../tests/data/tutorial.fil\")" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "myFil" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`myFil` now contains an instance of the :class:`sigpyproc.readers.FilReader` class. We can access obervational\n", "metadata through the `myFil.header` attribute:" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
Header(\n",
       "    filename='../../tests/data/tutorial.fil',\n",
       "    data_type='filterbank',\n",
       "    nchans=64,\n",
       "    foff=-1.09,\n",
       "    fch1=1510.0,\n",
       "    nbits=2,\n",
       "    tsamp=0.00032,\n",
       "    tstart=50000.0,\n",
       "    nsamples=187520,\n",
       "    nifs=1,\n",
       "    coord=<SkyCoord (ICRS): (ra, dec) in deg\n",
       "    (0., 0.)>,\n",
       "    azimuth=<Angle 0. deg>,\n",
       "    zenith=<Angle 0. deg>,\n",
       "    telescope='Fake',\n",
       "    backend='FAKE',\n",
       "    source='P: 250.000000000000 ms, DM: 30.000',\n",
       "    frame='topocentric',\n",
       "    ibeam=0,\n",
       "    nbeams=0,\n",
       "    dm=0,\n",
       "    period=0,\n",
       "    accel=0,\n",
       "    signed=False,\n",
       "    rawdatafile=None,\n",
       "    hdrlens=[244],\n",
       "    datalens=[3000320],\n",
       "    filenames=['../../tests/data/tutorial.fil'],\n",
       "    nsamples_files=[187520],\n",
       "    tstart_files=[50000.0]\n",
       ")\n",
       "
\n" ], "text/plain": [ "\u001b[1;35mHeader\u001b[0m\u001b[1m(\u001b[0m\n", " \u001b[33mfilename\u001b[0m=\u001b[32m'../../tests/data/tutorial.fil'\u001b[0m,\n", " \u001b[33mdata_type\u001b[0m=\u001b[32m'filterbank'\u001b[0m,\n", " \u001b[33mnchans\u001b[0m=\u001b[1;36m64\u001b[0m,\n", " \u001b[33mfoff\u001b[0m=\u001b[1;36m-1.09\u001b[0m,\n", " \u001b[33mfch1\u001b[0m=\u001b[1;36m1510\u001b[0m\u001b[1;36m.0\u001b[0m,\n", " \u001b[33mnbits\u001b[0m=\u001b[1;36m2\u001b[0m,\n", " \u001b[33mtsamp\u001b[0m=\u001b[1;36m0\u001b[0m\u001b[1;36m.00032\u001b[0m,\n", " \u001b[33mtstart\u001b[0m=\u001b[1;36m50000\u001b[0m\u001b[1;36m.0\u001b[0m,\n", " \u001b[33mnsamples\u001b[0m=\u001b[1;36m187520\u001b[0m,\n", " \u001b[33mnifs\u001b[0m=\u001b[1;36m1\u001b[0m,\n", " \u001b[33mcoord\u001b[0m=\u001b[1m<\u001b[0m\u001b[1;95mSkyCoord\u001b[0m\u001b[39m \u001b[0m\u001b[1;39m(\u001b[0m\u001b[39mICRS\u001b[0m\u001b[1;39m)\u001b[0m\u001b[39m: \u001b[0m\u001b[1;39m(\u001b[0m\u001b[39mra, dec\u001b[0m\u001b[1;39m)\u001b[0m\u001b[39m in deg\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[1;39m(\u001b[0m\u001b[1;36m0\u001b[0m\u001b[39m., \u001b[0m\u001b[1;36m0\u001b[0m\u001b[39m.\u001b[0m\u001b[1;39m)\u001b[0m\u001b[1m>\u001b[0m,\n", " \u001b[33mazimuth\u001b[0m=\u001b[1m<\u001b[0m\u001b[1;95mAngle\u001b[0m\u001b[39m \u001b[0m\u001b[1;36m0\u001b[0m\u001b[39m. deg\u001b[0m\u001b[1m>\u001b[0m,\n", " \u001b[33mzenith\u001b[0m=\u001b[1m<\u001b[0m\u001b[1;95mAngle\u001b[0m\u001b[39m \u001b[0m\u001b[1;36m0\u001b[0m\u001b[39m. deg\u001b[0m\u001b[1m>\u001b[0m,\n", " \u001b[33mtelescope\u001b[0m=\u001b[32m'Fake'\u001b[0m,\n", " \u001b[33mbackend\u001b[0m=\u001b[32m'FAKE'\u001b[0m,\n", " \u001b[33msource\u001b[0m=\u001b[32m'P: 250.000000000000 ms, DM: 30.000'\u001b[0m,\n", " \u001b[33mframe\u001b[0m=\u001b[32m'topocentric'\u001b[0m,\n", " \u001b[33mibeam\u001b[0m=\u001b[1;36m0\u001b[0m,\n", " \u001b[33mnbeams\u001b[0m=\u001b[1;36m0\u001b[0m,\n", " \u001b[33mdm\u001b[0m=\u001b[1;36m0\u001b[0m,\n", " \u001b[33mperiod\u001b[0m=\u001b[1;36m0\u001b[0m,\n", " \u001b[33maccel\u001b[0m=\u001b[1;36m0\u001b[0m,\n", " \u001b[33msigned\u001b[0m=\u001b[3;91mFalse\u001b[0m,\n", " \u001b[33mrawdatafile\u001b[0m=\u001b[3;35mNone\u001b[0m,\n", " \u001b[33mhdrlens\u001b[0m=\u001b[1m[\u001b[0m\u001b[1;36m244\u001b[0m\u001b[1m]\u001b[0m,\n", " \u001b[33mdatalens\u001b[0m=\u001b[1m[\u001b[0m\u001b[1;36m3000320\u001b[0m\u001b[1m]\u001b[0m,\n", " \u001b[33mfilenames\u001b[0m=\u001b[1m[\u001b[0m\u001b[32m'../../tests/data/tutorial.fil'\u001b[0m\u001b[1m]\u001b[0m,\n", " \u001b[33mnsamples_files\u001b[0m=\u001b[1m[\u001b[0m\u001b[1;36m187520\u001b[0m\u001b[1m]\u001b[0m,\n", " \u001b[33mtstart_files\u001b[0m=\u001b[1m[\u001b[0m\u001b[1;36m50000.0\u001b[0m\u001b[1m]\u001b[0m\n", "\u001b[1m)\u001b[0m\n" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "Pretty(myFil.header)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "where" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "sigpyproc.header.Header" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "type(myFil.header)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "All values stored in the `myFil.header` attribute may be accessed as attributes, i.e.:" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "64" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "myFil.header.nchans" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now that we know how to load a file into `sigpyproc`, let’s look at at doing something with the loaded data." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Dedispersing the data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "One of the most used techniques in pulsar processing is dedispersion, wherein we add or remove frequency dependent\n", "time delays to the data.\n", "\n", "To dedisperse our `myFil` instance, we simply call the dedisperse method:" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "e6efb548100d49e99f7b00fd712a5fbd", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Output()" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
\n"
      ],
      "text/plain": []
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "
\n",
       "
\n" ], "text/plain": [ "\n" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "myTim = myFil.dedisperse(30)" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "TimeSeries([108., 100., 102., ..., 105., 111., 107.], dtype=float32)" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "myTim" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "sigpyproc.timeseries.TimeSeries" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "type(myTim)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here we have dedispersed to a DM of 30 pc cm$^{-3}$ with the result being an instance of the\n", ":autoclass:`sigpyproc.timeseries.TimeSeries` class, which we have called `myTim`.\n", "\n", "The :autoclass:`sigpyproc.timeseries.TimeSeries` class in a subclass of `numpy.ndarray`, and is capable of using\n", "all standard numpy functions. For example:" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "TimeSeries(19636992., dtype=float32)" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "myTim.sum()" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "TimeSeries(121., dtype=float32)" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "myTim.max()" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "TimeSeries(88., dtype=float32)" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "myTim.min()" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "TimeSeries(105., dtype=float32)" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.median(myTim)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The use of `numpy.ndarray` subclasses is important in allowing sigpyproc to easily interface with many 3rd party\n", "python libraries." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Performing a Fourier transform" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To perform a discrete fourier transform of the data contained in the `myTim` instance we may invoke the `myTim.rFFT`\n", "method." ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "myFS = myTim.rfft()" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "sigpyproc.fourierseries.FourierSeries" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "type(myFS)" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "FourierSeries([ 1.9636884e+07 +0.j , -2.9424850e+02 +429.87863j,\n", " 5.4652838e+02 -577.44696j, ...,\n", " -1.3942198e+03+1670.1677j , -1.2117781e+03-1779.332j ,\n", " -2.7670000e+03 +0.j ], dtype=complex64)" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "myFS" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The :autoclass:`sigpyproc.fourierseries.FourierSeries` is also a subclass of `numpy.ndarray`, where array elements are `numpy.complex64`.\n", "\n", "Using the `remove_rednoise` method of `myFS`, we can de-redden the Fourier series:" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [], "source": [ "myFS_red = myFS.remove_rednoise()" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "FourierSeries([ 1. +0.j , nan +nanj,\n", " nan +nanj, ..., -0.8724377+1.045113j,\n", " -0.7582742-1.113423j, -1.7314595+0.j ],\n", " dtype=complex64)" ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "myFS_red" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "with the dereddened fourier series, we can now form the power spectrum of the observation:" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [], "source": [ "mySpec = myFS_red.form_spec(interpolated=True)" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "PowerSpectrum([1. , nan, nan, ..., 1.3613995, 1.5284487,\n", " 1.7314595], dtype=float32)" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "mySpec" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here we have set the `interpolated` flag to True, causing the `formSpec` function to perform nearest bin interpolation.\n", "\n", "`mySpec` contains several convenience methods to help with navigating the power spectrum. For instance:" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "240" ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" } ], "source": [ "mySpec.period2bin(0.25)" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "300" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "mySpec.freq2bin(5.0)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can also perofrm Lyne-Ashworth harmonic folding to an arbitrary number of harmonics:" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [], "source": [ "folds = mySpec.harmonic_fold(5)" ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[PowerSpectrum([1. , nan, nan, ..., 2.7979643, 2.9650135,\n", " 1.7314595], dtype=float32),\n", " PowerSpectrum([1. , nan, nan, ..., 2.7979643, 2.9650135,\n", " 1.7314595], dtype=float32),\n", " PowerSpectrum([1. , nan, nan, ..., 2.7979643, 2.9650135,\n", " 1.7314595], dtype=float32),\n", " PowerSpectrum([1. , nan, nan, ..., 2.7979643, 2.9650135,\n", " 1.7314595], dtype=float32),\n", " PowerSpectrum([1. , nan, nan, ..., 2.7979643, 2.9650135,\n", " 1.7314595], dtype=float32)]" ] }, "execution_count": 24, "metadata": {}, "output_type": "execute_result" } ], "source": [ "folds" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Where the variable `folds` is a python list containing each of the requested harmonic folds." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Folding data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Both the :autoclass:`sigpyproc.timeseries.TimeSeries` and the :autoclass:`sigpyproc.fourierseries.FourierSeries`\n", "have methods to phase fold their data. Using our earlier myFil instance, we will fold our filterbank file with a period\n", "of 250 ms and a DM of pc cm$^{-3}$ and acceleration of 0 ms$^{-2}$." ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "64a13bef55324e1fb97eb996d7468347", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Output()" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
\n"
      ],
      "text/plain": []
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "
\n",
       "
\n" ], "text/plain": [ "\n" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "myFold = myFil.fold(0.25,30.,accel=0,nbins=128,nints=32,nbands=16)" ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "sigpyproc.foldedcube.FoldedData" ] }, "execution_count": 26, "metadata": {}, "output_type": "execute_result" } ], "source": [ "type(myFold)" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(32, 16, 128)" ] }, "execution_count": 27, "metadata": {}, "output_type": "execute_result" } ], "source": [ "myFold.shape" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The the :autoclass:`sigpyproc.foldedcube.FoldedData` has several functions to enable simple slicing and summing of\n", "the folded data cube. These include:\n", "\n", "* `getSubband`: select all data in a single frequency band\n", "* `getSubint`: select all data in a single subintegration\n", "* `getFreqPhase`: sum the data in the time axis\n", "* `getTimePhase`: sum the data in the frequency axis\n", "*`getProfile`: get the pulse profile of the fold\n", "\n", "We can also tweak the DM and period of the fold using the `updateParams` method:" ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [], "source": [ "myFold.update_dm(dm=100)\n", "myFold.update_period(period=0.2502)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Tips and tricks" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "There are several tips and tricks to help speed up `sigpyproc` and also make it more user friendly. For people who are\n", "familiar with Python and IPython these will be old news, but for newbies these may be of use." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Tab completion**: One of the many nice things about IPython is that it allows for tab completion:" ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 30, "metadata": {}, "output_type": "execute_result" } ], "source": [ "myFil # then press tab" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Docstrings**: by using question marks or double question marks we can access both information about a function and\n", "its raw source:" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [], "source": [ "myFil.downsample?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that all docstrings are written in `numpydoc`. This is to facilitate automatic documentation creation with the\n", "Sphinx package." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Chaining**: The ability to chain together methods, combined with history recall in IPython means that it is simple to\n", "condense a sigpyproc request into a single line:" ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "ff74b23ce7794699b5bd31fab49275de", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Output()" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
\n"
      ],
      "text/plain": []
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "
\n",
       "
\n" ], "text/plain": [ "\n" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "spectrum = FilReader(\"../../tests/data/tutorial.fil\").collapse().rfft().remove_rednoise().form_spec(True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here we create a `FilReader` instance, which is then collapsed in frequency, FFT’d, cleaned of rednoise and interpolated to form a power spectrum. In the intrests of readability, this is not always a good idea, however for testing code\n", "quickly, it is invaluable." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.12" } }, "nbformat": 4, "nbformat_minor": 4 }