Add option for machine readable output.

[AndreasPK]

It would be great to have a way to get runtime results as csv to feed it into scripts and whatnot.

[nobrakal]

I have added for myself a flag

  -r,--saveRawResults ARG  Write the raw results in a given file

Which basically write the data-results (a big list) in a file using Haskell's show. It can certainly be upgraded to something more useful !

Do you prefer csv, json or yaml ? Is there any better option ?

[AndreasPK]

I think csv in a manner like criterion ouputs would be easiest to use.

[nobrakal]

@AndreasPK Sadly csv is not very adapted for recursive data structure (my main structure is recursive) so I have chosen JSON.

A sample result is:

{
  "graphsArgs":[
    [
      "Mesh",
      3
    ],
    [
      "Clique",
      3
    ]
  ],
  "benchResults":[
    [
      "vertexList",
      {
        "tag":"Group",
        "contents":[
          {
            "tag":"Group",
            "contents":[
              {
                "tag":"Group",
                "contents":[
                  {
                    "tag":"Simple",
                    "contents":[
                      "",
                      [
                        [
                          "Containers",
                          [
                            1.1240917632327363e-6,
                            4.667235798336615e-8
                          ]
                        ],
                        [
                          "Alga",
                          [
                            1.1239781062541625e-5,
                            5.27687839543705e-7
                          ]
                        ],
                        [
                          "Fgl",
                          [
                            4.517922303008385e-6,
                            2.387000976828053e-7
                          ]
                        ],
                        [
                          "Hash-Graph",
                          [
                            2.329615860039123e-6,
                            1.6582005052838644e-8
                          ]
                        ]
                      ]
                    ]
                  }
                ]
              }
            ]
          },
          {
            "tag":"Group",
            "contents":[
              {
                "tag":"Group",
                "contents":[
                  {
                    "tag":"Simple",
                    "contents":[
                      "",
                      [
                        [
                          "Containers",
                          [
                            1.0915197195973146e-6,
                            1.917957456760297e-8
                          ]
                        ],
                        [
                          "Alga",
                          [
                            3.734580401806883e-4,
                            4.069230576938172e-6
                          ]
                        ],
                        [
                          "Fgl",
                          [
                            4.374144645456693e-6,
                            6.472984118445801e-8
                          ]
                        ],
                        [
                          "Hash-Graph",
                          [
                            2.357397684647263e-6,
                            5.007456033250274e-8
                          ]
                        ]
                      ]
                    ]
                  }
                ]
              }
            ]
          }
        ]
      }
    ]
  ]
}

obtained from time run -o vertexList -r raw.json.

It is using the Grouped data type because it is the one I am using internally. You can find it in BenchGraph.Render.Types

The two numbers for each pass is the computed time by criterion and the standard deviation.

What do you think ? Do you want:

• More detailed info about benchmarks ?
• Analysis results (like the comparison between these numbers) ?

The produced JSON can also be used to produce graphs directly.

[nobrakal]

I merged for now, since this is useful in this state.

Please let me know if you want more features :)

[AndreasPK]

As I said criterion style csv would be easiest to use. But I guess I can make json work :)