-
Notifications
You must be signed in to change notification settings - Fork 37
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Dockerfile + README - Add basic support for YARN #7
Comments
I'm looking at this and there's a chicken and egg problem. YARN node manager needs to see the YARN resource manager on startup to be able to register. I start the resource manager then I start the node manager with -link to link the node manager to the resource manager. When I do that I can see the node registering to the resource manager. However when you submit a job, the resource manager needs to communicate with the node manager. I cannot link the node manager when I start the resource manager since the node manager does not exist yet. I could try start both service within the same container... |
teragen (1GB)
|
Nice... so how does this container connects to YARN ? On Thu, Sep 24, 2015 at 2:03 PM, Tony Bussières notifications@github.com
|
Docker 1.8 :-) On Thu, Sep 24, 2015 at 4:09 PM, David Lauzon notifications@github.com
|
BTW It works with --name and not -h
All the docker containers that runs in the same docker host auto-populates
/etc/hosts with the NAME of the containers an their IP
|
Awesome!! uh, it works on the --name? So we may not need the -h parameter anymore On Thu, Sep 24, 2015 at 4:22 PM, Tony Bussières notifications@github.com
|
Using -h is still a good practice IMO and you need it if you continue to
use --link.
However you can specify only --name to use the /etc/host "auto populate"
feature.
|
Perfect! It means we'll be able to simplify the README. On Sat, Sep 26, 2015 at 8:09 PM, Tony Bussières notifications@github.com
|
Add basic support to run YARN using this image (if not already required by #6).
Make the minimal changes to the Dockerfile and add instructions in the README for running an example mapreduce job (included in hadoop source code) on top of YARN.
The text was updated successfully, but these errors were encountered: