Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add mega-analysis vs. meta-analysis example #59

Open
tsalo opened this issue Dec 9, 2020 · 2 comments · May be fixed by #74
Open

Add mega-analysis vs. meta-analysis example #59

tsalo opened this issue Dec 9, 2020 · 2 comments · May be fixed by #74
Labels
documentation Improvements or additions to documentation

Comments

@tsalo
Copy link
Member

tsalo commented Dec 9, 2020

@jdkent and I were working on a data exercise comparing mega- and meta-analysis for the ABCD-ReproNim course, and I was thinking that the general approach of taking a multi-site dataset, running a random intercepts model taking site into account, and comparing that to a "meta-analysis" where each site is treated as its own study would be a good addition to the PyMARE documentation.

Here is a basic attempt at this: https://github.com/tsalo/misc-notebooks/blob/master/run_mega_and_meta.ipynb

@tsalo tsalo added the documentation Improvements or additions to documentation label Dec 9, 2020
@tyarkoni
Copy link
Contributor

tyarkoni commented Dec 9, 2020

That's cool to illustrate the conceptual point that the two are different, though I'm not convinced it makes sense to add as a PyMARE example. Generally speaking, if you have access to the raw data from different sites, and the measurements are commensurable, you're better off doing the former—so you probably wouldn't be looking to use PyMARE if you had that kind of data. But I could be convinced. I think you add some text describing the relevant factors in deciding which approach to use—e.g., that mega-analysis is generally better, but that if the measures/scales differ across sites, and you lack a good way to model that, you might want to do a meta-analysis instead—that might help.

You might also want to add random slopes to the mega-analysis model, as that seems conceptually a bit closer to the meta-analysis model.

@tsalo
Copy link
Member Author

tsalo commented Dec 9, 2020

Agreed. It's key to show that meta-analysis is inferior to mega-analysis. I could add text to make it clear that the meta-analysis version reflects a version where you are pulling those coefficients from different papers rather than a central dataset.

You might also want to add random slopes to the mega-analysis model, as that seems conceptually a bit closer to the meta-analysis model.

I hadn't thought of that. I think random intercepts would be enough for the mega-analysis for ABCD-ReproNim, but for this example I can include random slopes.

@tsalo tsalo linked a pull request Dec 18, 2021 that will close this issue
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants