Merge branch 'dev' into contributor-lr

.github/BuildAndTest.md

@@ -0,0 +1,183 @@
+## Building OWA EPANET From Source on Windows
+by Michael E. Tryby
+
+Created on: March 13, 2018 
+
+
+### Introduction
+
+Building OWA's fork of EPANET from source is a basic skill that all developers 
+interested in contributing to the project should know how to perform. This 
+document describes the build process step-by-step. You will learn 1) how to 
+configure your machine to build the project locally; 2) how to obtain the 
+project files using git; 3) how to use cmake to generate build files and build 
+the project; and 4) how to use ctest and nrtest to perform unit and regression 
+testing on the build artifacts produced. Be advised, you will need local admin 
+privileges on your machine to follow this tutorial. Let’s begin! 
+
+### Dependencies
+
+Before the project can be built the required tools must be installed. The OWA 
+EPANET project adheres to a platform compiler policy - for each platform there 
+is a designated compiler. The platform compiler for Windows is Visual 
+Studio cl, for Linux gcc, and for Mac OS clang. These instructions describe how
+to build EPANET on Windows. CMake is a cross platform build, testing, and packaging 
+tool that is used to automate the EPANET build workflow. Boost is a free portable 
+peer-reviewed C++ library. Unit tests are linked with Boost unit test libraries. 
+Lastly, git is a free and open source distributed version control system. Git must 
+be installed to obtain the project source code from the OWA EPANET repository 
+found on GitHub. 
+
+### Summary of Build Dependencies
+  - Platform Compiler
+    - Windows: Visual Studio 10.0 32-bit cl (version 16.00.40219.01 for 80x86)
+  - CMake (version 3.0.0 or greater)
+  - Boost Libraries (version 1.58 or greater)
+  - git (version 2.6.0 or greater)
+
+### Build Procedure
+1. Install Dependencies
+  * A. Install Visual Studio 2010 Express and SP1
+    Our current benchmark platform and compiler is Windows 32-bit Visual Studio 10 
+    2010. Older versions of Visual Studio are available for download here:  
+
+    https://www.visualstudio.com/vs/older-downloads/ 
+
+    A service pack for Visual Studio 10 2010 is available here: 
+
+    https://www.microsoft.com/en-us/download/details.aspx?id=34677
+
+  * B. Install Boost 
+    Boost binaries for Windows offer a convenient installation solution. Be sure to 
+    select for download the boost installer exe that corresponds to the version of Visual Studio you have installed. 
+
+    https://sourceforge.net/projects/boost/files/boost-binaries/1.58.0/
+
+    Although newer version of Boost are available, a link to Boost 1.58 is provided. This is the library version that the unit tests have been written against. Older versions of Boost may not work. The default install location for the Boost 
+    Libraries is C:\local\boost_1_58_0
+
+  * C. Install Chocolatey, CMake, and git
+    Chocolatey is a Windows package manager that makes installing some of these 
+    dependencies a little easier. When working with Chocolatey it is useful to have 
+    local admin privileges. Chocolatey is available here:
+
+    https://chocolatey.org/install
+
+    Once Chocolately is installed, from a command prompt running with admin privileges 
+    issue the following commands
+    ```
+    \>choco install -y cmake --installargs 'ADD_CMAKE_TO_PATH=User'
+    \>choco install -y git --installargs /GitOnlyOnPath
+    \>refreshenv
+    ```
+
+  * D. Common Problems
+    Using chocolatey requires a command prompt with admin privileges. 
+    Check to make sure installed applications are on the command path. 
+    Make note of the Boost Library install location. 
+
+
+2. Build The Project
+  As administrator open a Visual Studio 2010 Command Prompt. Change directories to 
+  the location where you wish to build the EPANET project. Now we will issue a series 
+  of commands to create a parent directory for the project root and clone the project
+  from OWA's GitHub repository.  
+
+  * A. Clone the EPANET Repository
+    ```
+    \>mkdir OWA
+    \>cd OWA
+    \>git clone --branch=dev https://github.com/OpenWaterAnalytics/EPANET.git
+    \>cd EPANET
+    ```
+  The present working directory is now the project root EPANET. The directory contains 
+  the same files that are visibly present in the GitHub Repo by browsing to the URL
+  https://github.com/OpenWaterAnalytics/EPANET/tree/dev. 
+
+  Now we will create a build products directory and generate the platform build 
+  file using cmake.  
+
+  * B. Generate the build files
+    ```
+    \>mkdir buildprod
+    \>cd buildprod
+    \>set BOOST_ROOT=C:\local\boost_1_58_0
+    \>cmake -G "Visual Studio 10 2010" -DBOOST_ROOT="%BOOST_ROOT%" -DBoost_USE_STATIC_LIBS="ON" ..
+    ```
+  
+  Now that the dependencies have been installed and the build system has been 
+  generated, building EPANET is a simple CMake command. 
+
+  * C. Build EPANET
+    \>cmake --build . --config Debug
+    
+  * D. Common Problems
+    CMake may not be able to find the project CMakeLists.txt file or the Boost 
+    library install location. 
+
+
+3. Testing
+  Unit Testing uses Boost Unit Test library and CMake ctest as the test runner. 
+  Cmake has been configured to register tests with ctest as part of the build process. 
+
+  * A. Unit Testing
+    ```
+    \>cd tests
+    \>ctest -C Debug
+    ```
+  The unit tests run quietly. Ctest redirects stdout to a log file which can be 
+  found in the "tests\Testing\Temporary" folder. This is useful when a test fails.  
+
+  Regression testing is somewhat more complicated because it relies on Python 
+  to execute EPANET for each test and compare the binary files and report files. 
+  To run regression tests first python and any required packages must be installed. 
+  If Python is already installed on your local machine the installation of 
+  miniconda can be skipped.  
+
+  * B. Installing Regression Testing Dependencies
+    ```
+    cd ..\..
+    \>choco install -y miniconda --installargs '/AddToPath=1'
+    \>choco install -y curl
+    \>choco install -y 7zip 
+    \>refreshenv
+    \>pip install -r tools/requirements-appveyor.txt
+    ```
+
+  With Python and the necessary dependencies installed, regression testing can be run 
+  using the before-test and run-nrtest helper scripts found in the tools folder. The script 
+  before-test stages the test and benchmark files for nrtest. The script run-nrtest calls 
+  nrtest execute and nrtest compare to perform the regression test. 
+
+  To run the executable under test, nrtest needs the absolute path to it and a 
+  unique identifier for it such as the version number. The project cmake build places build 
+  artifacts in the buildprod\bin\ folder. On Windows the build configuration "Debug" or 
+  "Release" must also be indicated. On Windows it is also necessary to specify the path to 
+  the Python Scripts folder so the nrtest execute and compare commands can be found. You 
+  need to substitute bracketed fields below like "<build identifier>" with the values for 
+  your setup. 
+
+  * C. Regression Testing
+    ```
+    \>tools\before-test.cmd <relative path to regression test location> <absolute path to exe under test> <build identifier>
+    \>tools\run-nrtest.cmd <absolute path to python scripts> <relative path to regression test location> <build identifier>
+    ```
+
+  * D. Common Problems
+  The batch file before-test.cmd needs to run with admin privileges. The nrtest script complains when it can't find manifest files. 
+
+That concludes this tutorial on building OWA EPANET from source on Windows. 
+You have learned how to configure your machine satisfying project dependencies 
+and how to acquire, build, and test EPANET on your local machine. To be sure, 
+there is a lot more to learn, but this is a good start! Learn more about project
+build and testing dependencies by following the links provided below. 
+
+### Further Reading
+  * Visual Studio - https://msdn.microsoft.com/en-us/library/dd831853(v=vs.100).aspx
+  * CMake - https://cmake.org/documentation/
+  * Boost - http://www.boost.org/doc/
+  * git - https://git-scm.com/doc
+  * Miniconda - https://conda.io/docs/user-guide/index.html
+  * curl - https://curl.haxx.se/
+  * 7zip - https://www.7-zip.org/
+  * nrtest - https://nrtest.readthedocs.io/en/latest/

include/epanet2.bas

@@ -56,6 +56,9 @@ Public Const EN_EFFICIENCY = 16
 Public Const EN_HEADCURVE = 17
 Public Const EN_EFFICIENCYCURVE = 18
 Public Const EN_PRICEPATTERN = 19
+Public Const EN_STATE = 20
+Public Const EN_CONST_POWER = 21
+Public Const EN_SPEED = 22
 
 Public Const EN_DURATION = 0      ' Time parameters
 Public Const EN_HYDSTEP = 1
@@ -159,6 +162,13 @@ Public Const EN_INITFLOW = 10    ' Re-initialize flow flag
 Public Const EN_CONST_HP = 0      ' constant horsepower
 Public Const EN_POWER_FUNC = 1    ' power function
 Public Const EN_CUSTOM = 2        ' user-defined custom curve
+Public Const EN_NOCURVE = 3       ' no curve
+
+Public Const EN_V_CURVE = 0       ' volume curve
+Public Const EN_P_CURVE = 1       ' pump curve
+Public Const EN_E_CURVE = 2       ' efficiency curve
+Public Const EN_H_CURVE = 3       ' head loss curve
+Public Const EN_G_CURVE = 4       ' General\default curve
 
 'These are the external functions that comprise the DLL
 
@@ -224,6 +234,7 @@ Public Const EN_CUSTOM = 2        ' user-defined custom curve
  Declare Function ENgetcurve Lib "epanet2.dll" (ByVal curveIndex As Long, ByVal CurveID As String, nValues As Long, xValues As Any, yValues As Any) As Long
  Declare Function ENgetheadcurveindex Lib "epanet2.dll" (ByVal pumpIndex As Long, curveIndex As Long) As Long
  Declare Function ENgetpumptype Lib "epanet2.dll" (ByVal index As Long, PumpType As Long) As Long
+ Declare Function ENgetcurvetype Lib "epanet2.dll" (ByVal curveindex As Long, CurveType As Long) As Long
 
  Declare Function ENgetversion Lib "epanet2.dll" (value As Long) As Long
 

include/epanet2.h

@@ -115,7 +115,10 @@ typedef enum {
   EN_EFFICIENCY   = 16,
   EN_HEADCURVE    = 17,
   EN_EFFICIENCYCURVE = 18,
-  EN_PRICEPATTERN = 19
+  EN_PRICEPATTERN = 19,
+  EN_STATE        = 20,
+  EN_CONST_POWER  = 21,
+  EN_SPEED        = 22
 } EN_LinkProperty;
 
 /// Time parameter codes
@@ -253,9 +256,17 @@ typedef enum {
 typedef enum {
   EN_CONST_HP    = 0,   /* constant horsepower       */
   EN_POWER_FUNC  = 1,   /* power function            */
-  EN_CUSTOM      = 2    /* user-defined custom curve */
+  EN_CUSTOM      = 2,   /* user-defined custom curve */
-} EN_CurveType;
+  EN_NOCURVE     = 3    /* no curve                  */
+} EN_PumpType;
 
+typedef enum {
+  EN_V_CURVE     = 0,   /*    volume curve                      */
+  EN_P_CURVE     = 1,   /*    pump curve                        */
+  EN_E_CURVE     = 2,   /*    efficiency curve                  */
+  EN_H_CURVE     = 3,   /*    head loss curve                   */
+  EN_G_CURVE     = 4    /*    General\default curve             */
+} EN_CurveType;
 
 // --- Declare the EPANET toolkit functions
 #if defined(__cplusplus)
@@ -777,10 +788,19 @@ extern "C" {
    @param linkIndex The index of the pump element
    @param[out] outType The integer-typed pump curve type signifier (output parameter)
    @return Error code
-   @see EN_CurveType
+   @see EN_PumpType
    */
   int  DLLEXPORT ENgetpumptype(int linkIndex, int *outType);
-  
+
+  /**
+   @brief Get the type of a curve
+   @param curveIndex The index of the curve element
+   @param[out] outType The integer-typed curve curve type signifier (output parameter)
+   @return Error code
+   @see EN_CurveType
+   */
+  int  DLLEXPORT ENgetcurvetype(int curveIndex, int *outType);
+    
   /**
    @brief Get the version number. This number is to be interpreted with implied decimals, i.e., "20100" == "2(.)01(.)00"
    @param[out] version The version of EPANET
@@ -1206,6 +1226,7 @@ extern "C" {
   int DLLEXPORT EN_getheadcurveindex(EN_Project *p, int pumpIndex, int *curveIndex);
   int DLLEXPORT EN_setheadcurveindex(EN_Project *p, int pumpIndex, int curveIndex);
   int DLLEXPORT EN_getpumptype(EN_Project *p, int linkIndex, int *outType);
+  int DLLEXPORT EN_getcurvetype(EN_Project *p, int curveIndex, int *outType);
   int DLLEXPORT EN_getversion(int *version);
   int DLLEXPORT EN_setcontrol(EN_Project *p, int cindex, int ctype, int lindex, EN_API_FLOAT_TYPE setting, int nindex, EN_API_FLOAT_TYPE level);
   int DLLEXPORT EN_setnodevalue(EN_Project *p, int index, int code, EN_API_FLOAT_TYPE v);

src/epanet.c

@@ -449,6 +449,10 @@ int DLLEXPORT ENgetpumptype(int index, int *type) {
   return EN_getpumptype(_defaultModel, index, type);
 }
 
+int DLLEXPORT ENgetcurvetype(int curveindex, int *type) {
+  return EN_getcurvetype(_defaultModel, curveindex, type);
+}
+
 int DLLEXPORT ENgetnumdemands(int nodeIndex, int *numDemands) {
   return EN_getnumdemands(_defaultModel, nodeIndex, numDemands);
 }
@@ -2046,6 +2050,7 @@ int DLLEXPORT EN_getlinknodes(EN_Project *p, int index, int *node1,
 int DLLEXPORT EN_getlinkvalue(EN_Project *p, int index, EN_LinkProperty code, EN_API_FLOAT_TYPE *value) {
   double a, h, q, v = 0.0;
   int returnValue = 0;
+  int pmp;
   
   EN_Network *net = &p->network;
   hydraulics_t *hyd = &p->hydraulics;
@@ -2059,8 +2064,6 @@ int DLLEXPORT EN_getlinkvalue(EN_Project *p, int index, EN_LinkProperty code, EN
   double *LinkFlows = hyd->LinkFlows;
   double *LinkSetting = hyd->LinkSetting;
 
-  
-  
   /* Check for valid arguments */
   *value = 0.0;
   if (!p->Openflag)
@@ -2176,6 +2179,38 @@ int DLLEXPORT EN_getlinkvalue(EN_Project *p, int index, EN_LinkProperty code, EN
       else
         v = 1.0;
       break;
+
+    case EN_STATE:
+      v = hyd->LinkStatus[index];
+      
+      if (Link[index].Type == EN_PUMP) {
+         pmp = findpump(net, index);
+         if (hyd->LinkStatus[index] >= OPEN) {
+            if (hyd->LinkFlows[index] > hyd->LinkSetting[index] * Pump[pmp].Qmax)
+               v = XFLOW;
+            if (hyd->LinkFlows[index] < 0.0)
+               v = XHEAD;
+         }
+      }
+      break;
+
+    case EN_CONST_POWER:
+      v = 0;
+      if (Link[index].Type == EN_PUMP) {
+         pmp = findpump(net, index);
+         if (Pump[pmp].Ptype == CONST_HP) {
+             v = Link[index].Km; // Power in HP
+         }
+      }
+      break;
+
+    case EN_SPEED:
+      v = 0;
+      if (Link[index].Type == EN_PUMP) {
+         pmp = findpump(net, index);
+         v = Link[index].Kc;
+      }
+      break;
       
     case EN_SETTING:
       if (Link[index].Type == EN_PIPE || Link[index].Type == EN_CVPIPE) {
@@ -2985,7 +3020,7 @@ int DLLEXPORT EN_addcurve(EN_Project *p, char *id) {
 
   strcpy(tmpCur[n].ID, id);
   tmpCur[n].Npts = 1;
-  tmpCur[n].Type = -1;
+  tmpCur[n].Type = G_CURVE;
   tmpCur[n].X = (double *)calloc(tmpCur[n].Npts, sizeof(double));
   tmpCur[n].Y = (double *)calloc(tmpCur[n].Npts, sizeof(double));
   if (tmpCur[n].X == NULL)
@@ -3026,9 +3061,7 @@ int DLLEXPORT EN_addcurve(EN_Project *p, char *id) {
 int DLLEXPORT EN_setcurve(EN_Project *p, int index, EN_API_FLOAT_TYPE *x, EN_API_FLOAT_TYPE *y, int n) {
   
   EN_Network *net = &p->network;  
-  Scurve *Curve = net->Curve;
+  Scurve *Curve = net->Curve;  
-  
-  
   int j;
 
   /* Check for valid arguments */
@@ -3059,12 +3092,9 @@ int DLLEXPORT EN_setcurve(EN_Project *p, int index, EN_API_FLOAT_TYPE *x, EN_API
 int DLLEXPORT EN_setcurvevalue(EN_Project *p, int index, int pnt, EN_API_FLOAT_TYPE x, EN_API_FLOAT_TYPE y) {
   
   EN_Network *net = &p->network;
-  
   Scurve *Curve = net->Curve;
-  
   const int Ncurves = net->Ncurves;
   
-  
   if (!p->Openflag)
     return (102);
   if (index <= 0 || index > Ncurves)
@@ -3375,7 +3405,8 @@ int DLLEXPORT EN_setheadcurveindex(EN_Project *p, int index, int curveindex) {
   pump->Q0 /= Ucf[FLOW];
   pump->Qmax /= Ucf[FLOW];
   pump->Hmax /= Ucf[HEAD];
-
+  
+  p->network.Curve[curveindex].Type = P_CURVE;
   return (0);
 }
 
@@ -3396,6 +3427,18 @@ int DLLEXPORT EN_getpumptype(EN_Project *p, int index, int *type) {
   return (0);
 }
 
+int DLLEXPORT EN_getcurvetype(EN_Project *p, int curveindex, int *type) {
+  
+  EN_Network *net = &p->network;
+    
+  if (!p->Openflag)
+    return (102);
+  if (curveindex < 1 || curveindex > net->Ncurves)
+    return (206);
+  *type = net->Curve[curveindex].Type;
+  return (0);
+}
+
 /*
 ----------------------------------------------------------------
    Functions for opening files
@@ -3765,7 +3808,7 @@ int allocdata(EN_Project *p)
     }
     for (n = 0; n <= par->MaxCurves; n++) {
       net->Curve[n].Npts = 0;
-      net->Curve[n].Type = -1;
+      net->Curve[n].Type = G_CURVE;
       net->Curve[n].X = NULL;
       net->Curve[n].Y = NULL;
     }

src/types.h

@@ -151,8 +151,9 @@ typedef enum {
   V_CURVE,       /*    volume curve                      */
   P_CURVE,       /*    pump curve                        */
   E_CURVE,       /*    efficiency curve                  */
-  H_CURVE
+  H_CURVE,       /*    head loss curve                   */
-} CurveType;     /*    head loss curve                   */
+  G_CURVE        /*    General\default curve             */
+} CurveType;
 
 typedef enum {
   CONST_HP,      /*    constant horsepower              */

tools/nrtest-epanet/main.py

@@ -4,7 +4,9 @@ import time
 import cStringIO
 import itertools as it
 
-import epanet_reader as er
+# project import
+import nrtest_epanet.output_reader as er
+
 
 def result_compare(path_test, path_ref, comp_args):
 
@@ -15,29 +17,36 @@ def result_compare(path_test, path_ref, comp_args):
     total = 0
     output = cStringIO.StringIO()
     eps =  np.finfo(float).eps
+    min_cdd = 100.0
     
     start = time.time()
     
-    test_reader = er.reader(path_test)
+    test_reader = er.output_generator(path_test)
-    ref_reader = er.reader(path_ref)
+    ref_reader = er.output_generator(path_ref)
     
     for test, ref in it.izip(test_reader, ref_reader):
         total += 1
         if total%100000 == 0:
             print(total)
         
-        if test.size != ref.size:
+        if len(test[0]) != len(ref[0]):
             raise ValueError('Inconsistent lengths')
         
         # Skip results if they are zero or equal
-        if np.array_equal(test, ref):
+        #if np.array_equal(test, ref):
-            equal += 1
+        #    equal += 1
-            continue
+        #    continue
         else:
             try:
-                np.testing.assert_allclose(test, ref, 1.0e-06, 2*eps)
+                diff = np.fabs(np.subtract(test[0], ref[0]))
-                close += 1
+                idx = np.unravel_index(np.argmax(diff), diff.shape)
                 
+                if diff[idx] != 0.0:
+                    tmp = - np.log10(diff[idx])
+                    
+                    if tmp < min_cdd:
+                        min_cdd = tmp;
+                        
             except AssertionError as ae:
                 notclose += 1
                 output.write(str(ae))
@@ -49,8 +58,9 @@ def result_compare(path_test, path_ref, comp_args):
     print(output.getvalue())
     output.close()
     
-    print('equal: %d  close: %d  notclose: %d  total: %d  in %f (sec)\n' % 
+    print('mincdd: %d in %f (sec)' % (np.floor(min_cdd), (stop - start)))
-          (equal, close, notclose, total, (stop - start)))
+    #print('equal: %d  close: %d  notclose: %d  total: %d  in %f (sec)\n' % 
+    #      (equal, close, notclose, total, (stop - start)))
 
     if notclose > 0:
         print('%d differences found\n' % notclose)
@@ -120,6 +130,8 @@ def nrtest_execute(app_path, test_path, output_path):
     exit(not success)
 
 
+import report_diff as rd
+
 if __name__ == "__main__":
 #    app_path = "apps\\swmm-5.1.11.json"
 #    test_path = "tests\\examples\\example1.json"
@@ -130,7 +142,10 @@ if __name__ == "__main__":
 #    ref_path  = "C:\\Users\\mtryby\\Workspace\\GitRepo\\Local\\epanet-testsuite\\benchmarks\\v2012"
 #    print(nrtest_compare(test_path, ref_path, (0.001, 0.0)))
 
-    
+    benchmark_path = "C:\\Users\\mtryby\\Workspace\\GitRepo\\michaeltryby\\epanet-lr\\nrtestsuite\\benchmarks\\"
-    path_test = "C:\\Users\\mtryby\\Workspace\\GitRepo\\Local\\epanet-testsuite\\benchmarks\\v2011a\\Example_3\\example3.out"
+    path_test = benchmark_path + "epanet-220dev\\example2\\example2.out"
-    path_ref  = "C:\\Users\\mtryby\\Workspace\\GitRepo\\Local\\epanet-testsuite\\benchmarks\\v2012\\Example_3\\example3.out"    
+    path_ref  = benchmark_path + "epanet-2012\\example2\\example2.out"
-    result_compare(path_test, path_ref, (0.001, 0.0))
+
+    #result_compare(path_test, path_ref, (0.001, 0.0))
+    rd.report_diff(path_test, path_ref, 2)
+    

tools/nrtest-epanet/nrtest_epanet/__init__.py

@@ -28,7 +28,7 @@ __copyright__ = "None"
 __credits__ = "Colleen Barr, Maurizio Cingi, Mark Gray, David Hall, Bryant McDonnell"
 __license__ = "CC0 1.0 Universal"
 
-__version__ = "0.4.0"
+__version__ = "0.5.0"
 __date__ = "September 6, 2017"
 
 __maintainer__ = "Michael Tryby"
@@ -38,7 +38,7 @@ __status  = "Development"
 
 def epanet_allclose_compare(path_test, path_ref, rtol, atol):
     ''' 
-    Compares results in two EPANET binary files. Using the comparison criteria 
+    Compares results in two EPANET binary files using the comparison criteria 
     described in the numpy assert_allclose documentation. 
             
         (test_value - ref_value) <= atol + rtol * abs(ref_value) 
@@ -67,22 +67,67 @@ def epanet_allclose_compare(path_test, path_ref, rtol, atol):
     for (test, ref) in it.izip(ordr.output_generator(path_test), 
                                ordr.output_generator(path_ref)):
         
-        if len(test) != len(ref):
+        if len(test[0]) != len(ref[0]):
             raise ValueError('Inconsistent lengths')
         
         # Skip over arrays that are equal
-        if np.array_equal(test, ref):
+        if np.array_equal(test[0], ref[0]):
             continue
         else:
-            np.testing.assert_allclose(test, ref, rtol, atol)
+            np.testing.assert_allclose(test[0], ref[0], rtol, atol)
 
     return True
 
-# def epanet_better_compare(path_test, path_ref, rtol, atol):
+
-#     '''
+def epanet_mincdd_compare(path_test, path_ref, rtol, atol):
-#     If you don't like assert_allclose you can add another function here. 
+    '''
-#     '''
+    Compares the results of two EPANET binary files using a correct decimal
-#     pass
+    digits (cdd) comparison criteria:
+
+           min cdd(test, ref) >= atol
+
+    Returns true if min cdd in the file is greater than or equal to atol,
+    otherwise an AssertionError is thrown.
+
+    Arguments:
+        path_test - path to result file being testedgit
+        path_ref  - path to reference result file
+        rtol - ignored
+        atol - minimum allowable cdd value (i.e. 3)
+
+    Returns:
+        True
+
+    Raises:
+        ValueError()
+        AssertionError()
+    '''
+    min_cdd = 100.0
+
+    for (test, ref) in it.izip(ordr.output_generator(path_test), 
+                               ordr.output_generator(path_ref)):
+
+        if len(test[0]) != len(ref[0]):
+            raise ValueError('Inconsistent lengths')
+
+        # Skip over arrays that are equal
+        if np.array_equal(test[0], ref[0]):
+            continue
+        else:
+            diff = np.fabs(np.subtract(test[0], ref[0]))
+            idx = np.unravel_index(np.argmax(diff), diff.shape)
+
+            if diff[idx] != 0.0:
+                tmp = - np.log10(diff[idx])
+
+                if tmp < min_cdd:
+                    min_cdd = tmp;
+
+    if np.floor(min_cdd) >= atol:
+        return True
+    else:
+        raise AssertionError('min_cdd=%d less than atol=%g' % (min_cdd, atol))
+
 
 def epanet_report_compare(path_test, path_ref, rtol, atol):
     '''

tools/nrtest-epanet/nrtest_epanet/output_reader.py

@@ -23,7 +23,8 @@ def output_generator(path_ref):
     yield element attributes. It is useful for comparing contents of binary 
     files for numerical regression testing. 
     
-    The generator yields a Python list containing element attributes. 
+    The generator yields a Python tuple containing an array of element 
+    attributes and a tuple containing the element type, period, and attribute. 
     
     Arguments: 
         path_ref - path to result file
@@ -38,7 +39,8 @@ def output_generator(path_ref):
             for element_type in oapi.ElementType:
                 for attribute in br.elementAttributes[element_type]:
         
-                    yield br.element_attribute(element_type, period_index, attribute)
+                    yield (br.element_attribute(element_type, period_index, attribute), 
+                           (element_type, period_index, attribute))
 
 
 class OutputReader():    

tools/nrtest-epanet/report_diff.py

@@ -0,0 +1,96 @@
+# -*- coding: utf-8 -*-
+
+#
+#  report_diff.py 
+# 
+#  Date Created: July 11, 2018
+#
+#  Author:     Michael E. Tryby
+#              US EPA - ORD/NRMRL
+#
+
+# system imports
+import itertools as it
+
+# third party imports
+import numpy as np
+
+# project imports
+import nrtest_epanet.output_reader as ordr
+
+
+def _binary_diff(path_test, path_ref, min_cdd):
+    for (test, ref) in it.izip(ordr.output_generator(path_test), 
+                               ordr.output_generator(path_ref)):
+        
+        if len(test[0]) != len(ref[0]):
+            raise ValueError('Inconsistent lengths')
+        
+        # Skip over arrays that are equal
+        if np.array_equal(test[0], ref[0]):
+            continue
+        else:
+            lre = _log_relative_error(test[0], ref[0])
+            idx = np.unravel_index(np.argmin(lre), lre.shape)
+
+            if lre[idx] < min_cdd: 
+                _print_diff(idx, lre, test, ref)
+
+    return 
+
+
+def _log_relative_error(q, c):
+    '''
+    Computes log relative error, a measure of numerical accuracy. 
+    
+    Single precision machine epsilon is between 2^-24 and 2^-23.
+    
+    Reference:
+    McCullough, B. D. "Assessing the Reliability of Statistical Software: Part I."
+    The American Statistician, vol. 52, no. 4, 1998, pp. 358-366. 
+    '''
+    diff = np.subtract(q, c)
+    tmp_c = np.copy(c)
+    # If ref value is small compute absolute error
+    tmp_c[np.fabs(tmp_c) < 1.0e-6] = 1.0
+    
+    re = np.fabs(diff)/np.fabs(tmp_c)
+    # If re is tiny set lre to number of digits
+    re[re < 1.0e-7] = 1.0e-7
+    # If re is very large set lre to zero
+    re[re > 2.0] = 1.0
+
+    return np.negative(np.log10(re))
+    
+    
+def _print_diff(idx, lre, test, ref):
+    
+    idx_val = (idx[0], ref[1])
+    test_val = (test[0][idx[0]])
+    ref_val = (ref[0][idx[0]])
+    diff_val = (test_val - ref_val)
+    lre_val = (lre[idx[0]])
+    
+    print("Idx: %s\nSut: %f  Ref: %f  Diff: %f  LRE: %f\n" 
+          % (idx_val, test_val, ref_val, diff_val, lre_val))
+
+
+def report(args):
+    _binary_diff(args.test, args.ref, args.mincdd)
+
+
+if __name__ == '__main__':
+    from argparse import ArgumentParser
+    
+    parser = ArgumentParser(description='EPANET benchmark difference reporting')
+    parser.set_defaults(func=report)
+    parser.add_argument('-t', '--test', default=None, 
+                        help='Path to test benchmark')
+    parser.add_argument('-r', '--ref', default=None, 
+                        help='Path to reference benchmark')
+    parser.add_argument('-mc', '--mincdd', type=int, default=3, 
+                        help='Minimum correct decimal digits')
+    
+    args = parser.parse_args()
+    args.func(args)
+    

tools/nrtest-epanet/setup.py

@@ -17,6 +17,7 @@ except ImportError:
 entry_points = {
     'nrtest.compare': [
         'epanet allclose = nrtest_epanet:epanet_allclose_compare',
+        'epanet mincdd = nrtest_epanet:epanet_mincdd_compare',
         'epanet report = nrtest_epanet:epanet_report_compare',
         # Add entry point for new comparison functions here
     ]
@@ -24,7 +25,7 @@ entry_points = {
 
 setup(
     name='nrtest-epanet',
-    version='0.4.0',
+    version='0.5.0',
     description="EPANET extension for nrtest",
     
     author="Michael E. Tryby",

win_build/WinSDK/epanet2.def

@@ -94,4 +94,6 @@ EXPORTS
     ENdeletenode                  = _ENdeletenode@4
     ENsetlinktype                 = _ENsetlinktype@8
     ENgetdemandmodel              = _ENgetdemandmodel@16
-    ENsetdemandmodel              = _ENsetdemandmodel@16
+    ENsetdemandmodel              = _ENsetdemandmodel@16
+    ENgetcurvetype                = _ENgetcurvetype@8
+