-
Notifications
You must be signed in to change notification settings - Fork 39
trappy/ftrace: merge primary and secondary DFs based on pivot #260
base: master
Are you sure you want to change the base?
Conversation
trappy/utils.py
Outdated
@@ -102,3 +105,55 @@ def handle_duplicate_index(data, | |||
dup_index_left += 1 | |||
|
|||
return data.reindex(new_index) | |||
|
|||
def merge_dfs(pr_df, sec_df, pivot): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add a doctsring for this method and explain with an example in the documentation.
(A short excerpt like the one you have in the gist.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok will do
trappy/utils.py
Outdated
try: | ||
if np.isnan(value): | ||
data[key] = pivot_map[data[pivot]][key] | ||
except: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A general except is not good. Can you specifically catch the excpetions that can possibly occur (and you expcect them to occur)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I got rid of this exception altogether to fix a perf issue and will post the fix soon, thanks
trappy/utils.py
Outdated
@@ -102,3 +105,55 @@ def handle_duplicate_index(data, | |||
dup_index_left += 1 | |||
|
|||
return data.reindex(new_index) | |||
|
|||
def merge_dfs(pr_df, sec_df, pivot): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, since this is core functionality. We would need tests as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep this is a bit of a preview so I didn't add yet. Will do, thanks
Forward propogate secondary DF into primary DF and return the merged DF. Implements: #250 Change-Id: I312d77302bbca8bb13bfa598785ebc0cc879fe34 Signed-off-by: Joel Fernandes <joelaf@google.com>
Signed-off-by: Joel Fernandes <joelaf@google.com>
This allows passing of arguments to callback, needed for the residency analysis state machine in LISA. Also update docstring. Signed-off-by: Joel Fernandes <joelaf@google.com>
ipynb/examples: Update Executor example
I hacked something up that's working, just posting here. Basic idea is concatenate the DFS, mark them as primary/secondary, walk through them and remember the secondary information, propagate it to primary and rebuild the primary DF. I haven't written test cases yet. (CC @derkling @bjackman @sinkap)
Here's a gist showing it working in @derkling 's notebook:
https://gist.github.com/joelagnel/cc08ba964e40467e828741c691011ffc