Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option for machine readable output. #10

Open
AndreasPK opened this issue Sep 2, 2018 · 5 comments
Open

Add option for machine readable output. #10

AndreasPK opened this issue Sep 2, 2018 · 5 comments
Assignees

Comments

@AndreasPK
Copy link
Contributor

It would be great to have a way to get runtime results as csv to feed it into scripts and whatnot.

@nobrakal
Copy link
Collaborator

nobrakal commented Sep 3, 2018

I have added for myself a flag

  -r,--saveRawResults ARG  Write the raw results in a given file

Which basically write the data-results (a big list) in a file using Haskell's show. It can certainly be upgraded to something more useful !

Do you prefer csv, json or yaml ? Is there any better option ?

@nobrakal nobrakal self-assigned this Sep 3, 2018
@AndreasPK
Copy link
Contributor Author

I think csv in a manner like criterion ouputs would be easiest to use.

@nobrakal
Copy link
Collaborator

@AndreasPK Sadly csv is not very adapted for recursive data structure (my main structure is recursive) so I have chosen JSON.

A sample result is:

{
  "graphsArgs":[
    [
      "Mesh",
      3
    ],
    [
      "Clique",
      3
    ]
  ],
  "benchResults":[
    [
      "vertexList",
      {
        "tag":"Group",
        "contents":[
          {
            "tag":"Group",
            "contents":[
              {
                "tag":"Group",
                "contents":[
                  {
                    "tag":"Simple",
                    "contents":[
                      "",
                      [
                        [
                          "Containers",
                          [
                            1.1240917632327363e-6,
                            4.667235798336615e-8
                          ]
                        ],
                        [
                          "Alga",
                          [
                            1.1239781062541625e-5,
                            5.27687839543705e-7
                          ]
                        ],
                        [
                          "Fgl",
                          [
                            4.517922303008385e-6,
                            2.387000976828053e-7
                          ]
                        ],
                        [
                          "Hash-Graph",
                          [
                            2.329615860039123e-6,
                            1.6582005052838644e-8
                          ]
                        ]
                      ]
                    ]
                  }
                ]
              }
            ]
          },
          {
            "tag":"Group",
            "contents":[
              {
                "tag":"Group",
                "contents":[
                  {
                    "tag":"Simple",
                    "contents":[
                      "",
                      [
                        [
                          "Containers",
                          [
                            1.0915197195973146e-6,
                            1.917957456760297e-8
                          ]
                        ],
                        [
                          "Alga",
                          [
                            3.734580401806883e-4,
                            4.069230576938172e-6
                          ]
                        ],
                        [
                          "Fgl",
                          [
                            4.374144645456693e-6,
                            6.472984118445801e-8
                          ]
                        ],
                        [
                          "Hash-Graph",
                          [
                            2.357397684647263e-6,
                            5.007456033250274e-8
                          ]
                        ]
                      ]
                    ]
                  }
                ]
              }
            ]
          }
        ]
      }
    ]
  ]
}

obtained from time run -o vertexList -r raw.json.

It is using the Grouped data type because it is the one I am using internally. You can find it in BenchGraph.Render.Types

The two numbers for each pass is the computed time by criterion and the standard deviation.

What do you think ? Do you want:

  • More detailed info about benchmarks ?
  • Analysis results (like the comparison between these numbers) ?

The produced JSON can also be used to produce graphs directly.

@nobrakal
Copy link
Collaborator

I merged for now, since this is useful in this state.

Please let me know if you want more features :)

@AndreasPK
Copy link
Contributor Author

As I said criterion style csv would be easiest to use. But I guess I can make json work :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants