README.md 3.51 KB
Newer Older
Sergey Karpov's avatar
Sergey Karpov committed
1 2
# STDPipe - Simple Transient Detection Pipeline
*AKA: random codes noone else will ever use*
Nicolas Leroy's avatar
Nicolas Leroy committed
3

Sergey Karpov's avatar
Sergey Karpov committed
4 5 6 7 8 9 10 11 12 13 14 15 16 17
*STDPipe* is a set of Python routines for astrometry, photometry and transient detection related tasks, intended for quick and easy implementation of custom pipelines, as well as for interactive data analysis.

### Design principles
 - implemented as a library of routines covering most common tasks
 - operates on standard Python objects: NumPy arrays for images, Astropy Tables for catalogs and object lists, etc
 - does not try to re-implement the things already implemented in other Python packages
 - conveniently wraps external codes that do not have their own Python interfaces (*SExtractor*, *SCAMP*, *PSFEx*, *HOTPANTS*, *Astrometry.Net*, ...)
     - wrapping is transparent: all data passed from Python, all options customizable from Python, all (or most of) outputs available back
     - everything operates on temporary files, nothing is kept after the run unless explicitly asked for

### Features
 - ~~pre-processing~~ - should be handled before in an instrument-specific way
     - bias/dark subtraction, flatfielding, masking
 - object detection and photometry
karpov-sv's avatar
karpov-sv committed
18
     - SExtractor or SEP for detection, photutils for photometry
Sergey Karpov's avatar
Sergey Karpov committed
19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37
 - astrometric calibration
     - Astrometry.Net for blind WCS solving
     - SCAMP or Astropy-based code for refinement
 - photometric calibration
     - Vizier catalogues, passband conversion (PS1 to Johnson, Gaia to Johnson, ...)
     - spatial polynomial + color term + intrinsic scatter
 - image subtraction
     - HiPS templates
     - PanSTARRS DR1 templates
     - hotpants + custom noise model
 - transient detection and photometry
     - noise-weighted detection, cutout adjustment, ...
 - auxiliary functions
     - PSF estimation, simulated stars, FITS header utilities, plotting, ...
 - light curve creation (soon)
     - spatial clustering, color regression, variability analysis, ...

# Installation

Sergey Karpov's avatar
Sergey Karpov committed
38
*STDpipe* is available at https://github.com/karpov-sv/stdpipe and is mirrored at https://gitlab.in2p3.fr/icare/stdpipe
Sergey Karpov's avatar
Sergey Karpov committed
39

karpov-sv's avatar
karpov-sv committed
40
The package is in constant development, so to keep track of the changes the suggested way of installing it is by cloning the repository
Sergey Karpov's avatar
Sergey Karpov committed
41
```
karpov-sv's avatar
karpov-sv committed
42
git clone https://github.com/karpov-sv/stdpipe.git
Sergey Karpov's avatar
Sergey Karpov committed
43
```
karpov-sv's avatar
karpov-sv committed
44
and then installing from it in development (or "editable") mode by running the command
Sergey Karpov's avatar
Sergey Karpov committed
45 46 47
```
python3 setup.py develop --user
```
karpov-sv's avatar
karpov-sv committed
48
This way you may update the repository or apply local patches, and it will immediately be reflected in the installed package.
karpov-sv's avatar
karpov-sv committed
49

Sergey Karpov's avatar
Sergey Karpov committed
50 51 52 53 54

Apart of Python requirements that will be installed automatically, *STDPipe* also (optionally) makes use of the following external software:
 - [SExtractor](https://github.com/astromatic/sextractor)
 - [SCAMP](https://github.com/astromatic/scamp)
 - [PSFEx](https://github.com/astromatic/psfex)
55
 - [SWarp](https://github.com/astromatic/swarp)
Sergey Karpov's avatar
Sergey Karpov committed
56 57 58
 - [HOTPANTS](https://github.com/acbecker/hotpants)
 - [Astrometry.Net](https://github.com/dstndstn/astrometry.net)

karpov-sv's avatar
karpov-sv committed
59 60 61 62 63
Most of them may be installed from your package manager. E.g. on Debian or Ubuntu systems it may look like that:
```
sudo apt install sextractor scamp psfex swarp
```

karpov-sv's avatar
karpov-sv committed
64 65
You may also check more detailed installation instructions [here](https://stdpipe.readthedocs.io/en/latest/installation.html).

Sergey Karpov's avatar
Sergey Karpov committed
66 67
# Usage

karpov-sv's avatar
karpov-sv committed
68
The [documentation for *STDPipe*](https://stdpipe.readthedocs.io/) is now being actively written. In the meantime you may check the examples inside [notebooks/](notebooks/) folder, especially the [tutorial](notebooks/stdpipe_tutorial.ipynb) that demonstrates basic steps of a typical image processing.