Benutzer-Werkzeuge

Webseiten-Werkzeuge


modellingcomponents:filter

Filters

Filters are mathematical operations running on a timeserie as a whole. They can be used as a preprocessing step, applied to the raw measurment data, at the end of the processing as a postprocessing step or „temporarily“ for visualisation in a plot sheet.

There is a default set of filters implemented. The available filters are explained in the following chapters.

Every filter can have attributes. Some attributes are used from most of the filters. For some of these attributes default values are defined:

stepWidth        ="1/120"
samplingFrequency="120"
windowWidth      ="12"
noisyData        ="true"
cutOffFrequency  ="5"
zeroPhase        ="true"
sectionMinLength ="10" 
useBinValues     ="false"

The filter types and characteristics can be defined in the XML markerset/labelset files which define the data loaded for processing or which parts of the processed data should be written in an output file.

Custom filters can be added by an plugin architecture.

in pre-/postprocessing

The filter-elements, defining the filter charatericstics, can be used at different positions in the hierarchie of a labelset:

<?xml version="1.0" encoding="UTF-8"?>
<LabelSet>
 <Filter name="butterorder2" implName="default" cutOffFrequency="20"
         windowWidth="" stepWidth="" noisyData="true" samplingFrequency="1211"
         zeroPhase="true" sectionMinLength="3"/>  
 <Labels fileSuffix="c3d" mimeType="application/c3d-vicon">
    <Filter name="butterorder2" implName="default" cutOffFrequency="20"
            windowWidth="" stepWidth="" noisyData="true" samplingFrequency="1211"
            zeroPhase="true" sectionMinLength="3"/>  
    <Label name="ABC" comment="" flag="" firstDerivative=""
           secondDerivative="" thirdDerivative="" differentiations=""
           optional="" unchangeable="" mathType="">
       <Filter name="butterorder2" implName="default" cutOffFrequency="20"
               windowWidth="" stepWidth="" noisyData="true" samplingFrequency="1211"
               zeroPhase="true" sectionMinLength="3"/>  
    </Label>
 </Labels>
</LabelSet>

Definitions as the first child of the labelset-element are applied to all timeseries, as a child of an label-element are applied only to the timeserie the label corresponds to.

in the plot-sheet

The filter-elements defining the filter charatericstics, can be used also as children of the label-elements in a plot sheet definition.

<?xml version="1.0" encoding="iso-8859-1" ?>
<Sheet name="Bilateral shoulder abdcution">
  <Pages>
    <Page name="Mir 100"  rows="4" columns="3">
    ...
       <Diagram name="Height" 
           column="1"
           row="1"
           xunit="%"
           yunit="mm"
           diagramDescriptionDown="down"
           diagramDescriptionUp="up"
           plane="" 
           diagramType="xt" 
           diagramTitle="Height">
 
           <Label name="Height"
              label="Height">
              <Filter name="dft" implName="default"/>
           </Label>
       </Diagram>  

Comments

  • Till now, only in xt()- and xy()-plots timeseries are filterd.

Available Filter implementations

Poly4Diff

General

A differentiation method, implemented by application of a 4-th order interpolation polynom (5 points) based on a sliding window with width 5. Thaś why the minimum trial length is five.

Attributes

Attribute Default Description Required
stepWidth 1/120 [s] The time distance between succeeding frames in the unit seconds. No
samplingFrequency 120 [Hz] The sampling frequency in the unit Hz as an alternative parameter to stepWidth. No

References

  • Hildebrand, F.B. (1974). Introduction to Numerical Analysis, 2nd Edition, pp.111
  • Kreyszig, Erwin (1983). Advanced Engineering Mathematics, 5th Edition, pp.793
  • Yakowitz Sydney and Szidarovsky, Ferenc (1989). An Introduction to Numerical Computations, 2nd Edition, pp.18

NaNs

If the timeserie includes NaNs the differention is done for the parts between the NaNs. If these parts are less than five, the values of this range is set to NaN.

Differntiation with smooth (smoothdiff)

A differentiation method that simultaneously smoothes the data. This method is based on a sliding window technique. The data is approximated within the window with a quadratic polynomial. The coefficients of the quadratic polynomial were then used for calculating the derivative at the window's center (Rand and Stelmach, 2005).

Attributes

Attribute Default Description Required
samplingFrequency 120 [Hz] Scaling coeffiecient (In our case : sampling frequency) No
windowWidth 12 Number of points in sliding window No
noisyData false Ways to treat ends - false ⇒ Not so good for noisy waveforms, but good (best) for smooth ones. true ⇒ For noisy waveforms is better than 1; Flattens ends for smooth ones. No

References

  • The original C++ implementation is from Dr. Yuri Shimansky.
  • Thanks for Laetitia Fradet for the idea to use this filter and to have a look in a Matlab implementation of this filter.

Issues

  • The last index is set to NaN calculation of the diffential quotient needs the value for the index i+1.
  • completly broken in the current version

Chunk Mean (chunkmean)

Calculate the mean of a chunk of a timeserie.

The timeserie length must be a multiple of the windows size. The phase of the result is shifted by (window size) /2 frames to the front. The result timeserie length = (original timeserie length)/(window size). This reduction of the timeserie length makes this filter useless in most of the cases.

Attribute Default Description Required
windowWidth 12 Width of the window in which the mean is determined. No

WeightedAverage

Simple Low pass digital filter.

result[i] = (data[i-2] + 3d*data[i-1] + 4d*data[i] + 3d*data[i+1] + data[i+2])/12d

Butterworth and differentiating filter (butterdiff)

This filter determines the derivative based on a 4-th order interpolation polynom (poly4diff) after conversion of 0-values to NaN and a data smoothing by a butterworth filter with second order. This filter is used as default for calculation of derivatives, e.g. configured by the attribute „differentiations“ in the label of input labelset.

Attribute Default Unit Description Required
stepWidth 1/120 s No
samplingFrequency 120 Hz No
cutOffFrequency 5 Hz No
zeroPhase true If set to false, only foward direction filterering is done resulting in phase shifting. No
windowWidth 5 Width of the window which is moved over the timeserie. No
deleteBorderFrames 5 Sometimes the borders to ranges of NaN are noise. With this attribute a number of frames can be defined which are set to NaN on the border. No

References

  • Hildebrand, F.B. (1974). Introduction to Numerical Analysis, 2nd Edition, pp.111.
  • Kreyszig, Erwin (1983). Advanced Engineering Mathematics, 5th Edition, pp.793.
  • Yakowitz Sydney and Szidarovsky, Ferenc (1989). An Introduction to Numerical Computations, 2nd Edition, pp.18.

Butterworth second order (butterorder2)

Zero-phase forward and reverse (optional) butterworth filter of second order.

Attributre Default Unit Description Required
samplingFrequency 120 Hz No
cutOffFrequency 5 Hz No
windowWidth 5 No
sectionMinLength 10 Segments minimal count of frames. If the data contains segments with less frames a warning is given and the data of that segment is not filtered. No
zeroPhase false No

Discrete Fourier Transformation (dft/dft_bin)

Computes 1D Discrete Fourier Transform (DFT). Each DFT output value S[k] is represents the amplitude at frequency 2πkN. The multiplication by the exponential function effectively shifts s[n] down in frequency by 2πkN. The sum over N time samples can then be thought of as applying a decimating lowpass filter. So, effectively, the N outputs of the DFT represent the results of applying a bank of equally-spaced filters across s[n]'s frequency band, thus measuring „how much“ energy is present in various frequency bands in the input signal.

Assuming that a large-enough sample rate was used during the conversion to discrete-time, the result of the DFT will look a lot like the Fourier transform of the original signal s(t).The DFT is only applicable to finite-length signals, so in order to apply it, one must truncate the discrete-time signal s[n] to some finite length N, resulting in a potentially-shorter signal sN[n].

The result array contains the power specturm based on the dft() without the leading DC component for k=0:

spectrum[k]=Math.sqrt(Math.pow(data[k*2],2.0)+Math.pow(data[k*2+1],2.0));

The center frequencies for the bins are available, if the needed attributes are set:

Attribute Default Description Required
samplingFrequency Sample rate in [Hz], needed to calculate the center frequencies for the bins. No
useBinValues false If set to „true“ than a timeserie with the bin values are determined. This needs to set the samplingFrequency attribute too No

Comments

  • The result array with the power spectrum has half size as the input array. Thatś why this filter can not be used in the input- or output-labelsets. It is assumed, that this filter is usefull only in the plotsheet functionality. If there are new applications which needs this filter to be defined in input- or output-labelset the needed resampling code can be added easy.

GCVSPL (Woltring filter)

Natural B-spline data smoothing, using the Generalized Cross-Validation and Mean-Squared Prediction Error Criteria of Craven & Wahba (1979).

Attribute Default Description Required
order cubic linear, cubic, quintic, heptic No
derivative 0 0= no derivative, 1=first derivative, 2=second derivative, … No
cutoffFrequency auto cut off frequency in [Hz] or „auto“ for automatic determination No
errorVariance GCV With the default value „GCV“ the degree of smoothing to minimize the generalized cross validation function is determined. This is asymptotically the same as minimizing the true mean squared error (Craven & Wahba, 1979) or set error variance explicit as double value >= 0 No
interpolateGaps true If set to true, then gaps will be interpolated No

The following parameters are determined and can be printed in the log-file:

CutOffFrequency, if it is autodetected GeneralizedCrossValidationValue (WK(1) ) MeanSquaredResidual (WK(2) ) ResidualSumSquaresDimension (WK(3)) = Estimate of the number of degrees of freedom of the residual sum of squares per dataset, with 0.lt.WK(3).lt.N-M. SmoothingParameter (WK(4)) = Smoothing parameter p, multiplicative with the splines' derivative constraint. TrueMeanSquareError (WK(5) ) = Estimate of the true mean squared error (different formula for |MD| = 3). GaussMarkovErrorVariance (WK(6))

MEMO: GCVSPL software package (C) COPYRIGHT 1985, 1986: H.J. Woltring

For large datasets (N » 0) and negligible boundary artefacts, the behaviour of a natural spline approximates that of a periodic spline. For the latter case, the frequency characteristic in the equidistantly sampled, uniformly weighted case is that of a double, phase-symmetric Butterworth filter, with transfer function H(w) = [1 + (w/wo)^2M]^-1, where w is the frequency, wo = (p*T)^(-0.5/M) the filter's cut-off frequency, p the smoothing parameter, T the sampling interval, and 2M the order of the spline. If T is expressed in seconds, the frequen- cies are expressed in radians/second.

It has been found empirically, that the effective number of estimated spline parameters Np is related to the Butterworth cut-off frequency wo as Np ~ M/2 + KM * wo * N * T, where Np ranges between M and N, and where KM is the integral over x from 0 to infinity of (1 + x^2M)^- divided by PI. For large M, KM approaches 1/PI from above; values for small M are: K1 = 1/2, K2 = 1/V8, K3 = 1/3. This relation has also been found to apply for uniformly weighted data which are sampled slightly anequidistantly, with T taken as the average sampling inter-val. For large Np, the relation with wo * N * T becomes nonlinear.

zero2nan

This filter is used as a workaround for data collection by a vicon system where it can happen that all three components of a position are equal 0 which mean that the data is wrong. The filter detects this case and sets the data point values to NaN.

moverms

defaultemg

bandpass

rectification

removeoffset

oneeuro

This is a simple lowpassfilter especially usefull for filtering humon motion data.

To minimize jitter and lag when tracking human motion, the two parameters (minCutOffFrequency and beta) can be set using a simple two-step procedure.

First beta is set to 0 and minCutOffFrequency to a reasonable middle-ground value such as 1 Hz. Then the body part is held steady or moved at a very low speed while minCutOffFrequency is adjusted to remove jitter and preserve an acceptable lag during these slow movements (decreasing minCutOffFrequency reduces jitter but increases lag, minCutOffFrequency must be > 0).

Next, the body part is moved quickly in different directions while beta is increased with a focus on minimizing lag. First find the right order of magnitude to tune beta, which depends on the kind of data you manipulate and their units: do not hesitate to start with values like 0.001 or 0.0001. You can first multiply and divide beta by factor 10 until you notice an effect on latency when moving quickly. Note that parameters minCutOffFrequency and beta have clear conceptual relationships: if high speed lag is a problem, increase beta; if slow speed jitter is a problem, decrease minCutOffFrequency.

Attribute Default Unit Description Required
samplingFrequency 120 Hz No
minCutOffFrequency 5 Hz No
beta 1 Hz No
dCutOffFrequency 0 Hz No

References

  • Casiez G., Roussel N., et.al. 1EUR Filter (2012). CHI'12, the 30th Conference on Human Factors in Computing Systems

resampling

Resampling with simple linear inper/extrapolation.

Attribute Default Unit Description Required
inFrequency 120 Hz No
outFrequency Hz Yes

If outFrequency > inFrequency the more frames at the end are cut, else the unavailable frames at the end are filled with NaN.

interpolation

Interpolation NaN/Inf-values with different possible methods.

Attribute Default Description
method linear Possible methods are dividedDifference, linear, loess, neville and spline.
Method References
loess Locally weighted regression and smoothing scatterplots, W.S. Cleveland, Journal of the American Statistical Association, Vol. 74, No. 368 (Dec., 1979), pp. 829-836.
neville Introduction to Numerical Analysis, ISBN 038795452X, chapter 2.
dividedDifference Introduction to Numerical Analysis, ISBN 038795452X, chapter 2. or Hildebrand, F. B. Introduction to Numerical Analysis. New York: McGraw-Hill, pp. 43-44 and 62-63, 1956.

NaN-values at the end of the timeserie can not be interpolated.

Adding filters via external "Math-Engines" like Matlab

modellingcomponents/filter.txt · Zuletzt geändert: 2020/07/06 10:48 von oliver