realtime.xml

<section id="SpectralRealTime">
  <title>Tools for Real-time Spectral Processing (pvs opcodes)</title>

  <para>
    With these opcodes,  two new core facilities are added to Csound. They offer improved audio quality, and fast performance, enabling high-quality analysis and resynthesis (together with transformations) to be applied in real-time to live signals. The original Csound phase vocoder remains unaltered; the new opcodes use an entirely separate set of functions based on <quote>pvoc.c</quote> in the CARL distribution, written by Mark Dolson. 
  </para>

  <para>
    The Csound <link linkend="dnoise"><citetitle>dnoise</citetitle></link> and <link linkend="src_conv"><citetitle>src_conv</citetitle></link> utilities (also by Dolson, from CARL) also use this pvoc engine. CARL pvoc is also the basis for the phase vocoder included in the Composer's Desktop Project. A few small but important modifications have been made to the original CARL code to support real-time streaming.
  </para>

  <para>
    <orderedlist>
      <listitem><para>Support for the new PVOC-EX analysis file format. This is a fully portable (cross-platform) open  file format, supporting three analysis formats, and multi-channel signals. Currently only the standard amplitude+frequency format has been implemented in the opcodes, but the file format itself supports amplitude+phase and  complex (real-imaginary) formats. In addition to the new opcodes, the original Csound pvoc opcodes have been extended (and thereby with enhanced audio quality in some cases)  to read PVOC-EX files as well as the original (non-portable) format.</para>

        <para>Full details of the structure of a PVOC-EX file are
        available via the website: <ulink
        url="http://www.cs.bath.ac.uk/~jpff/NOS-DREAM/researchdev/pvocex/pvocex.html"><citetitle>http://www.cs.bath.ac.uk/~jpff/NOS-DREAM/researchdev/pvocex/pvocex.html</citetitle></ulink>. This
        site also gives details of the freely available console programs
        pvocex and pvocex2 which can be used to create PVOC-EX files in
        all supported formats.
      </para></listitem> 

      <listitem><para>
        A new frequency-domain signal type, fully
        streamable, with <emphasis>f</emphasis> as the leading character.
        In this document it is conveniently referred to as an
        <emphasis>fsig</emphasis>. Primary support for fsigs is provided
        by the opcodes pvsanal and pvsynth, which perform conventional
        phase vocoder overlap-add analysis and resynthesis, independently
        of the orchestra control-rate. The only requirement is that the
        control-rate kr be higher than or equal to the analysis rate,
        whch can be expressed by the requirement that ksmps &lt;=
        overlap, where overlap is the distance in samples between
        analysis frames, as specified for pvsanal. As overlap is
        typically at least 128, and more usually 256, this is not an
        onerous restriction in practice. The opcode pvsinfo can be used
        at init time to acquire the properties of an fsig.</para>

      <para>The fsig enables the nominal separation between the
      analysis and resynthesis stages of the phase vocoder to be
      exposed to the Csound programmer, so that not only can
      alternatives be employed for either or both of these stages (not
      only oscillator-bank resynthesis, but also the generation of
      synthetic fsig streams), but opcodes, operating on the fsig
      stream, can themselves become more elemental. Thus the fsig
      enables the creation of a true streaming plugin framework for
      frequency domain signals. With the old pvoc opcodes, each opcode
      is required to act as a resynthesiser, so that facilities such
      as pitch scaling are duplicated in each opcode; and in many
      cases the opcodes are parameter-rich. The separation of analysis
      and synthesis stages by means of the fsig encourages the
      development of a wide range of simple building-block opcodes
      implementing one or two functions, with which more elaborate
      processes can be constructed.
      </para></listitem>
    </orderedlist>
  </para>

  <para>
    This is very much a preliminary and experimental release, and it is possible that the precise definition of the opcodes may change, in response to user feedback. Also, clearly, many new possibilities for opcodes are opened up; these factors may also have a retrospective influence on the opcodes presented here.
  </para>

  <para>
    Note that some opcode parameters currently have restricted or
    missing implementation. This is at least in part in order to keep
    the opcodes simple at this stage, and also because they highlight
    important design issues on which no decision has yet been made,
    and on which opinions from users are sought.
  </para>

  <para>
    One important point about the new signal type is that because the
    analysis rate is typically much lower than kr, new analysis frames
    are not available on each k-cycle. Internally, the opcodes track
    ksmps, and also maintain a frame counter, so that frames are read
    and written at the correct times; this process is generally
    transparent to the user. However, it means that k-rate signals
    only act on an fsig at the analysis rate, not at each k-cycle.
    The opocde pvsftw returns a k-rate flag that is set when new fsig
    data is valid.
  </para>

  <para>
    Because of the nature of the overlap-add system, the use of these
    opcodes incurs a small but significant delay, or latency,
    determined by the window size (max(ifftsize,iwinsize)). This is
    typically around 23msecs. In this first release, the delay is
    slightly in excess of the theoretical minimum, and it is hoped
    that it can be reduced, as the opcodes are further optimized for
    real-time streaming.
  </para>

  <para>
    The opcodes for real-time spectral processing are
    <link linkend="pvsadsyn"><citetitle>pvsadsyn</citetitle></link>,
    <link linkend="pvsanal"><citetitle>pvsanal</citetitle></link>,
    <link linkend="pvscross"><citetitle>pvscross</citetitle></link>,
    <link linkend="pvsfread"><citetitle>pvsfread</citetitle></link>,
    <link linkend="pvsftr"><citetitle>pvsftr</citetitle></link>,
    <link linkend="pvsftw"><citetitle>pvsftw</citetitle></link>,
    <link linkend="pvsinfo"><citetitle>pvsinfo</citetitle></link>,
    <link linkend="pvsmaska"><citetitle>pvsmaska</citetitle></link>, and
    <link linkend="pvsynth"><citetitle>pvsynth</citetitle></link>.
  </para>

  <para>
    In addition there are a number of opcodes available as plugins in
    Csound5,and in the core for Csound6.  These are
    <link linkend="pvstanal"><citetitle>pvstanal</citetitle></link>,
    <link linkend="pvsdiskin"><citetitle>pvsdiskin</citetitle></link>,
    <link linkend="pvscent"><citetitle>pvscent</citetitle></link>,
    <link linkend="pvsdemix"><citetitle>pvsdemix</citetitle></link>,
    <link linkend="pvsfreeze"><citetitle>pvsfreeze</citetitle></link>,
    <link linkend="pvsbuffer"><citetitle>pvsbuffer</citetitle></link>,
    <link linkend="pvsbufread"><citetitle>pvsbufread</citetitle></link>,
    <link linkend="pvsbufread2"><citetitle>pvsbufread2</citetitle></link>,
    <link linkend="pvscale"><citetitle>pvscale</citetitle></link>,
    <link linkend="pvshift"><citetitle>pvshift</citetitle></link>,
    <link linkend="pvsifd"><citetitle>pvsifd</citetitle></link>,
    <link linkend="pvsinit"><citetitle>pvsinit</citetitle></link>,
    <link linkend="pvsin"><citetitle>pvsin</citetitle></link>,
    <link linkend="pvsout"><citetitle>pvsout</citetitle></link>,
    <link linkend="pvsosc"><citetitle>pvsosc</citetitle></link>,
    <link linkend="pvsbin"><citetitle>pvsbin</citetitle></link>,
    <link linkend="pvsdisp"><citetitle>pvsdisp</citetitle></link>,
    <link linkend="pvsfwrite"><citetitle>pvsfwrite</citetitle></link>,
    <link linkend="pvsmix"><citetitle>pvsmix</citetitle></link>,
    <link linkend="pvsmooth"><citetitle>pvsmooth</citetitle></link>,
    <link linkend="pvsfilter"><citetitle>pvsfilter</citetitle></link>,
    <link linkend="pvsblur"><citetitle>pvsblur</citetitle></link>,
    <link linkend="pvstencil"><citetitle>pvstencil</citetitle></link>,
    <link linkend="pvsarp"><citetitle>pvsarp</citetitle></link>,
    <link linkend="pvsvoc"><citetitle>pvsvoc</citetitle></link>,
    <link linkend="pvsmorph"><citetitle>pvsmorph</citetitle></link>,
    <link linkend="pvsbandp"><citetitle>pvsbandp</citetitle></link>,
    <link linkend="pvsbandr"><citetitle>pvsbandr</citetitle></link>,
    <link linkend="pvswarp"><citetitle>pvswarp</citetitle></link>,
    <link linkend="pvsgain"><citetitle>pvsgain</citetitle></link>,
    <link linkend="pvs2tab"><citetitle>pvs2tab</citetitle></link>,
    <link linkend="tab2pvs"><citetitle>tab2pvs</citetitle></link>.
  </para>
  <para>
   A number of opcodes are designed to generate and process streaming
partials tracks data. these are
    <link linkend="partials"><citetitle>partials</citetitle></link>,
    <link linkend="trcross"><citetitle>trcross</citetitle></link>,
    <link linkend="trfilter"><citetitle>trfilter</citetitle></link>,
    <link linkend="trsplit"><citetitle>trsplit</citetitle></link>,
    <link linkend="trmix"><citetitle>trmix</citetitle></link>,
    <link linkend="trscale"><citetitle>trscale</citetitle></link>,
    <link linkend="trshift"><citetitle>trshift</citetitle></link>,
    <link linkend="trlowest"><citetitle>trlowest</citetitle></link>,
    <link linkend="trhighest"><citetitle>trhighest</citetitle></link>
    <link linkend="tradsyn"><citetitle>tradsyn</citetitle></link>,
    <link linkend="sinsyn"><citetitle>sinsyn</citetitle></link>,
    <link linkend="resyn"><citetitle>resyn</citetitle></link>,
    <link linkend="binit"><citetitle>binit</citetitle></link>

  </para>
  <para>
    See the <link linkend="ControlStacks"><citetitle>Stacks</citetitle> section </link> for information on the stack opcodes which can stack f-signals.
  </para>

</section>