Introduction

The Nimue platform is a set of modules to build up applications in the scientific field of motion analysis (gait analysis, sport sciences, robotics…). In the vocabulary of the Netbeans IDE the modules are bundeled as a platform.

The Nimue plaform includes implementations of a lot of mathematical algorithms for 3d analyses, graphical components for visualisation and also utilities to model the workflow of measurements and analyses. It is made for easy integration of measurement systems (markerbased optical-, inertial sensor based systems, …) and their specific data file formats.

Most parts of the Nimue platform will be made available open source again, via github repositories. I am working on transfering the code to github if I find time to do this. If this work is done I will add the links to the repositories into this documentation. It is a pity, but in the current version, the Nimue platform as a whole, is no longer available as a single repository, because of the restructureing of the codebase and due to licence issues of some of the modules. The reason to split the code into several repositories has to do with the need to have the parts of the platform with its own version numbering.

This is a draft version of the manual for version 2.2 of the Nimue Platform. It is far away from completeness. There are hundrets of html-pages which must be transfered to this Wiki. During this process I want to update its content and this needs a lot of time.

It gives you an overview about the main structure and features. You should read this document, if you are interested in developing motion analysis applications, based on the Nimue platform and if you want to learn about using specific features of your Nimue platform based motion analysis application, which are implemented as part of the platform.

This document is explicitly not a users manual and it's also not a manual for developers. If you have descided to develop an application based on the Nimue platform you have to ask for a developers guide. If you are working with an application based on the Nimue platform, better read first the applications manual.

The origin of the codebase comes from a project arranged by the Institue for Physic Didactics at the Karlsruhe Institute for Technoloogy. This was the work of Andreas Traupe and Oliver Rettig in the years 1998-1999.

The base idea was to use XML files to describe complex dynamic systems and the workflow of measurements and analyses for educational usage. The target application was called Skepsis. The easy processing possibilities of XML files allow simple automatisation and the use of powerful codegenerators and tools to develop graphical userinterfaces, where the XML files can be hidden for unexperienced users. With these experiences more generalized components are developed, to be able to build applications for research purposes.

In the following years many non mathematical orientated modules are added and a lot of non research applications are realized by the company ORAT Software-Entwicklung, which is founded as a spin off from the Karlsruhe Institute for Technology.

In the years 2003-2008 advanced components for motion analysis are added and the specialized components bundle Nimue for motion analysis was established. It was used for the application Upperlimb in the Gait- and Motionlab of the Heidelberg Orthopedic clinic for many years.

The extention of the framwork for inertial sensor based applications is started in Mai 2009 initially for the usage in the European community funded project Suregait. The idea of SureGait was to build up a low cost complete gait analysis system based on intertial and EMG sensors.

Since Sep. 2017 the Nimue platform is used in the RaHM-Lab for analysing robot motion at the DHBW Karlsruhe. Specific extentions to integrate with the ROS - robot operation system are implemented in JDataAquisition a Nimue platform based application with the focus on data aquisition for motion analysis.

Added

• Better documentation with examples of PointSetFit

Fixed

• Dataminer: Calculation of standard deviations failed
• DH Calc Element: modified dh and classical dh, index numbers start now at meaningfull positions (for modified dh alpha an r starts from 1 and d and theta starts from 0. For classical DH all parameters start with index 1.
• ransaclin filter does not work for input data including NaN-values. NaN-values are now automatically excluded. But keep in mind to adjust the parameters in these cases.

Added

• New functionality of the Forward Kinematics element which allows calculations based on modfied Denavit Hartenberg parameters

Fixed

• <DH>-Element now corretly determines the position trajectories

Added

• Method getFramesOfEachIntervall() extended to be able to get frames also from phases
• Added functionality to DenavitHartenberg Calc element to determine mean/std of d,r,alpha and theta values directly as parameters calculated by the element itself.
• Added warning messages if a filter results in a complete trajectory with at minimum of one component is NaN for each frame
• Addes „lastD“ parameter to the DH-element to define the TCP coordinate system by translation about this parameter around the z-axis of the last joint axis.
• Element to calculate forward kinematics based on Denavit Hartenberg convention

Fixed

• Added test for all values to be equal of 0.0 of matrix-3d objects and set them them to Double.MINVALUE if not all values are exatly 0.0 to exclude them for substitution by the Vicon WORKAROUND filter isersoSelective-Filter.
• Wrong ResouceBundle name „javax.vecmath.ExceptionString“ in vecmath: VecMathI18N.java
• Fixed constructor of DualVector6e elements used in the DH-element.

Added

• New attribute „expectedPhasesCount“ added to the element „Phase“ which defines the expected count of phases. If the created count of phases does not fit this values defined by this attribute a warning is given.
• New element <DH> to determine several variants of DH-parameters from circle fits.
• New Datamining parameter Types: „std“ and „frames“
• Completion of RANSAC-Linear-Filter to preprocess static measurement-data to eliminate outliers.
• Support for filtering of quaternion-type timeseries during trial-loading for processing
• Added „fit“ parameter to ransaclin() to substitute excluded frames with the fit value instead of NaN.
• Additional attributes „minthreshold“, „maxthreashold“ to the <Event>-element, alow to use methods max/minvelocity in combination with „tolerance“.
• New attribute „invert“ added to the phase-element.

Fixed

• Loading RANSAC-Linear filter failed due to wrong specification of implementation class name
• Filterchain interation - only the first filter was used
• Event-Docu corrected

ToDo

• Filter zero2nan for matrix3x3, not only all zero but also unit-matrix with 1 in diagonal elements
• Workaround that fills all components of a matrix3d to NaN if ransaclin() has set only some of them to NaN
• if group names inclunde „_“ then they are excluded from the shown list in the DataExplorer

Added

• New function ddist(A,B,ref) to define a singed length of a vector based on a given reference vector.
• New property „seperateGroupFiles“ for the outputhandler MotionDataOutputHandler to save data for each calcgroup into sepearte files
• New function mdho() to determine the origin of a coordinate system based on modifed Denavit Hartenberg convention
• New function mdh() to determine the 4×4-Matrix of modified Denavit Hartenberg convention

Fixed

• The calculation element „LocalCoordinateSystem“ has not considered the includes/excludes attributes. Now the attributes can be used to prohibit calculation for specific trials.
• The <Phase>-Element has added wrong phases in the special case that there are single frames fulfilling the inphase-function. Theses 1-frame-length-phases are excluded now as it is specified.
• The <LocalCoordinateSystem>-Element overwrites the original given (by the attribute) global position. A new object is now created to prevent for this behavoir.
• The <GlobalCoordinateSystem>-element overwrites the original gien (by attribute) local position and rotation. New objects are now created to preent for this behavoir.
• The <Point>-Element used with calibration for specified phases does not work correct because the complete trial length is used instead only the number of frames from all phases together. Fixed by determing the the number of counts from the frames-object.
• poc()-function bug fixed

Added

• Timeseries read to be processed can be renamed defined by a mapping element in the input labelset. This was implemented explicit for GenericData, generic-c3d and vicon-c3d files only. The mapping functionality is now added for files in the d3d-file-format.
• New feature: Attribute „autodetectStaticFrames“ can be used to calibrate a point based on a non static trial by autodetection of static frames.
• New feature: The element <LocalCoordinateSystem> supports calibration.
• Documentation for the <GlobalFrame>-element in Show4d added.

Changed

• The axes calculated with AoR estimation based on the SARA algorithm are now normalized.
• The attribute „Rotation“ of the <CoordinateSystem>-element is interpreted as the name of a trajectory defining a matrix3d which descirbed the orientation of the coordinate system. This includes the automatically creation of alias trajectories for the columns of the matrix. This is changed to allow an arbitray formula describing the rotation. With this change alias trajectories are no longer created automatically.
• AoRSARA: Missing or not available movingCoordinateSystem/coordinateSystem is logged and not stops further processing

Added

• PointSetFit calc-element to fit two point sets based on singular value decomposition.
• Transpose function t() for 4×4 matrix type
• implemenation of iCoordinateSystem in the PointSetFit element to use the element name as a reference in the attributes of AoR-circle-fit calculation elements.
• The DataExplorer is now able to collect trial data from more than one motiondata file e.g. v3d and rtde, into one DataCollection, to be able to plot timeseries of all data files together inside one plot.
• PointSetFit: Possibility to use two different set of phases for each of the trajectories. This allows to work with trajectories which have different delays.
• PointSetFit: Automatically exclude phases for which one of the trajectories has no values.
• PointSetFit: New attribute „maxVariation“
• Export Data: The name of the phase defined as the labelgroup name is added as a suffix to the output filename.

Fixed

• AoRGamage2002 throws NullPointerException if used without the ReferencePoint attribute set.
• Created session parameters of the type Matrix3x3 are saved now if only generic xml file format is available to save session parameters.
• Sequences of trial types works now the same way as if the trial types are used separately, e.g. triggering works, configuration is read from the single trial types, etc.
• Constant types Vector3d, Matrix3d, Vector4d, Matrix4d and Quat4d are saved now properly in the session parameter file if the parameters are created by calibration. The functions vec(), vec4(), mat(), mat4() and q() are used.
• Catch NumberFormatException if in the Visual4d-View an undfined color name is used.
• If differentiation configuration is used in inputlabelsets to read data from multiple motiondata files warnings are thrown because of trying to create differentiations multiple times.

ToDo

• The axis estimation method <AoRSARA> needs the attribute „ReferencePoint“. The method <AoRGamage> works also without defining this attribute. This inconsistency should be eliminated. Change the code of AoRSARA to make the attribute „ReferencePoint“ optional.

Added

• Interpolation filter to fill NaN values based on a set of methods.

—————–

Added

• The method „percent“ added to the CalcModelElement <Event> to add events based on phase and the position at the given percentage of the width of the phase.
• Upgrade to Apache Netbeans 12 Platform
• new filter „resamling“

Fixed

• Refactoring FilterLabel to be able to add new properties without additional changes in LabelSetFilter class.

Added

• PolyDiff4 filter working with NaN values by application on NaN-free sections
• After processing data, a file is created only if a given labelset is not empty or not labelset is given. This behavoir is now implemented for all fileformats.
• New method frame for <Event>-Elements to define events at fix frame positions.
• New functionality with attributes „event“ and „offset“ to define events.
• Documentation about CalcGroup input properties added.
• Show4dView: Array definition based on „vector“ attribute is now possible without corresponnding „start“ attribute. If the „start“-attribute is missed (0,0,0) is used instead.

Fixed

• MotionData to trial converter imports double-timeseries as timeseries.
• butterorder2-filter with zero phase does not get NaN values in the first run
• Plot: NaN values are now shown as NaN, which means that the non NaN ranges are not connected
• Documentation of PlotSheet configuration: Better explaination of the usage of offsets.
• View4d: Not properly scaled arrows. The cone is now seperatly scaled with the inverse, so that scaling of cone plus cyclinder works fine.
• Export as csv with given labelset throws a null-pointer expception even if a labelGroup with the given labelGroup name exist but the filesuffix or mimetype is not configured. Fixed by checken of the mimetype and additionally of the filesuffix only if more than one labelgroup with the given name found.
• Export as csv with given labelset from selected datacollection inside a d3d-file results in saving with a file name of the d3d-file and the suffix == „*“ if no file suffix is given in the labelgroup file. Now the default suffix is „csv“ if no file suffix is explicitly set in the labelgroup definition and the name of the datacollection is used instead of the d3d-file name.

Planed

• If a dynamical calibration trial contains one or ore more incorrect points, it can happen that the functional calibration results in completly wrong joint center/axis determination. There is no way to exclude obviously wrong points. Maybe a session specific xml-file, which includes events and phases, can be read and manuelly set by the user.

Added

• ROS (robot operation system) added as a „Measurement System“. APIs are available to implement reading specific ROS measages
• Vicon-ROS-Wrapper integration
• ROS-OpenPose integration

Added

• New PlugIn mechanism to add plot diagram types.
• New diagram type „xyz“ to add 3d diagrams in the plot sheets.

Fixed

• The „processed data“ window at the bottom of the „data explorer“ window is now updated as expected in the following cases: 1. A session is processed and a new d3d and/or mp-file is created. 2. A file is externally deleted.

Added

• New Flag „allow gaps“ in the „saveMeanStd“ Action in the context menu of the plotsheet. If this flag is set, timeseries are not excluded any more if they contais NaN-values, as it is typically for adjunct-SHR curves.
• Warnings are shown, if „saveMeanStd“ or „save norm“ is invoked, but not data is available.

Changed

• <Phase>-Element used with condition-attribute is allowed without setting of the value-Attribut, optional minWidth/maxWidth attributes can be used.
• The CSVDataGroupExport Action now throws an IllegalArgumentException instead of null-PointExceptions in the cases of not allowed null arguments.
• A big refactoring has separeted the mathematical expression parser to make it pluggable via Service provider interface. This creates the precondition to plug in a parser subistitute, which can used GPU to increase the performance.

Removed

Fixed

• <Phase>-Element used with condition- and value-attribut throws execeptions if the value-timeseries contains NaN-value at positions insed the pahses defined by the condition.7
• <Phase>-Element used with condition-attribute optional checks incusion in given phases. The names of these phases are defined via phase-attribute instead former elementname + „Global“ suffix which does not make sense.
• If your try to export a dataGroup of a given d3d-file which does not conain any data a null-Pointer-Exception is thrown. The exception is catched now and a meaningful errormessage is given.
• Open a plot compoment by selecting a node which does not contain any data holds the plot-components tab with seeking for data sign open without any message. Fixed: Plot component and tab is closed and instead a warning dialog is shown.
• Export data from a selected group has expected a dataCollection as a direct child. It fails with an ArrayIndexOutOfBoundException, if the assumption is not fulfilled. Fixed by checking for dataCollection children first.

Added

• A new attribute „process“ in the <Real>-model element. This allows to move the calculation in the postprocessing phase, if the input variables used in the formula are available in the post processing phase only.

Changed

Removed

Fixed

• Some german language error messages from the joint center calculation code translated into english.

Planed

• The json export and also processing save as json, do both not save derivatives but should.

The following applications are build on top of the Nimue platform :

1. Skepsis:

This was the first application we are started with the company ORAT Software-Entwicklung. It was a distributed modelling system, for dynamica systems, focussed on teaching purposes. The application is not available any more.

2. Upperlimb:

This application is used from universities in Germany and in Swizerland and it is actively maintained by the company ORAT Software-Entwicklung.

3. GaitHdApp:

This application is made for usage in the Heidelberg Motionlab only but not established in the lab.

4. CMTGait:

This application is made for usage in the Heidelberg Motionlab only. It allows to record data with the sensors of the XSens measurement system. The data is processed into a structure that allows the application of the lab internal MoMo software for analysing gait. CMTGait not used any more in the lab because the XSens system is not used.

5. SureGaitApp:

This application was developed for the EU funded project SUREGAIT (FP6-SME, 32808).

6. LynxAnalyser:

This application is a generic motion data modelling and analysing tool and currently used for the projects MobMonRob and Ergobot in the RaHM-Lab at the DHBW Karlsruhe for robot motion analysis.

7. JDataAquisition:

This application is a generic data aquisition tool and currently used for the projects MobMonRob, Ergobot and KIRK in the RaHM-Lab at the DHBW Karlsruhe for robot motion analysis. It is possible to used it to collect data or human motion analysis based on data streamed by the Vicon Tracker software. It also includes a framework to collect data from the robot operation system (ROS).