Restore fb doc

website/docs/fb/ai-cluster.md

@@ -0,0 +1,20 @@
+---
+id: internal-fb-cluster
+title: Hydra on the internet FB Cluster
+---
+
+Support for launching jobs to the AI cluster is currently still experimental and is expected to evolve over
+the coming months.
+
+## flow-cli
+
+flow-cli integration is hacky at the moment.
+See the sample f6.sample_projects.classy_hydra_project.workflow.main for details.
+
+```bash title="Example run"
+$ CFG='{"config": {"overrides": ["trainer=multi_gpu","trainer.max_epochs=90","+lr_scheduler=multi_step"]}}'
+$ ENTITLEMENT=cv_images_gpu_prod
+$ TEAM=team_computer_vision
+$ WORKFLOW=f6.sample_projects.classy_hydra_project.workflow.main
+$ flow-cli canary $WORKFLOW --run-as-secure-group $TEAM --parameters-json=$CFG --entitlement $ENTITLEMENT
+```

website/docs/fb/configerator-config-source.md

@@ -0,0 +1,59 @@
+---
+id: fbcode-configerator-config-source
+title: Configerator Config Source Plugin
+---
+
+The ConfigeratorConfigSource plugin makes it possible for Hydra applications to consume a domain of configs from configerator.
+
+### Dependency
+
+Add the following to your `TARGET` file
+```commandline
+//fair_infra/fbcode_hydra_plugins/configerator_config_source:configerator_config_source
+```
+
+### Usage
+
+1. The Configerator Config Source plugin requires that you place the configs under a [domain](https://fburl.com/wiki/n5cgchxe).
+You can find an example domain [here](https://fburl.com/diffusion/ms50g5hu)
+
+:::important
+Due to the limitations of Configerator APIs, matching the name of your domain and directory of configs is necessary for the plugin to extract information on the full config names. 
+For example, the config paths returned by Configerator API could look like `fair_infra/hydra_plugins/configerator_config_source/example/db/mysql`. The plugin needs to know where the directory of configs begins ([`example`](https://fburl.com/diffusion/7c0c5tig)), in order to determine the full config name (`db/mysql`). So in this case the domain should be named [`example.cconf`](https://fburl.com/diffusion/pyymoo1t)
+:::
+
+
+2. Create a [SearchPathPlugin](https://hydra.cc/docs/next/advanced/search_path) to add the Configerator path to the list of search paths.
+   The path you add in your SearchPathPlugin should be the name of your domain of configs, such as in this [example SearchPathPlugin](https://fburl.com/diffusion/ljggtux5)
+
+:::info
+Adding a new search path will become much easier once [#274](https://github.com/facebookresearch/hydra/issues/274) is resolved, which is planned for Hydra 1.1.
+:::
+
+
+### Example:
+
+#### Example SearchPathPlugin
+[`ConfigeratorExampleSearchPathPlugin`](https://fburl.com/diffusion/vwa82fbg) adds the example configerator domain to the search path of the example applications.
+
+#### Reading primary config from configerator
+This example reads its primary config from configerator [here](https://fburl.com/diffusion/twk3smkj) which has a default list defined.
+
+```commandline
+$ buck run //fair_infra/fbcode_hydra_plugins/configerator_config_source/example/primary_config:my_app 
+...
+Parsing buck files: finished in 1.1 sec
+...
+{'driver': 'mysql', 'user': 'alau'}
+```
+
+#### Compose config with configerator
+This example reads its primary config from local yaml file `primary_config.yaml` but reads config groups info from configerator.
+
+```commandline
+$ buck run //fair_infra/fbcode_hydra_plugins/configerator_config_source/example/config_group:my_app -- +db=mysql
+...
+Parsing buck files: finished in 1.1 sec
+...
+{'foo': 'bar', 'driver': 'mysql', 'user': 'alau'}
+```

website/docs/fb/fair-cluster.md

@@ -0,0 +1,119 @@
+---
+id: fair-cluster
+title: Hydra on the FAIR cluster
+---
+
+import GithubLink from "@site/src/components/GithubLink"
+
+Hydra 1.0rc is available on FAIR Cluster. The recommended way for installation is via meta package [hydra-fair-plugin](https://github.com/fairinternal/hydra-fair-plugins).
+
+## Hydra FAIR Plugins
+1. It brings the correct Hydra dependency and has been tested on the FAIR Cluster.
+2. It provides FAIR Cluster specific defaults overrides (for example, ```hydra.sweep.dir``` is set to be ```/checkpoint/${oc.env:USER}/outputs/${now:%Y-%m-%d}/${now:%H-%M-%S}```)
+3. It provides a [fairtask](https://github.com/fairinternal/fairtask) launcher plugin.
+4. It installs [Submitit](https://github.com/facebookincubator/submitit) launcher plugin by default.
+
+### Installation
+<details>
+<summary>0.3.1 (stable), compatible with Hydra 0.11</summary>
+
+### Clean Install
+
+```commandline
+pip install hydra-fair-plugins
+```
+
+The dependency installed looks like
+```commandline
+$ pip freeze | grep hydra
+hydra-core==0.11.3
+hydra-fair-cluster==0.1.4
+hydra-fair-plugins==0.3.1
+hydra-fairtask==0.1.8
+hydra-submitit==0.2.0
+```
+</details>
+
+<details>
+<summary>1.0 (Release candidate), compatible with Hydra 1.0rc</summary>
+
+With [`Submitit`](https://github.com/facebookincubator/submitit) open sourced, the corresponding plugin has been moved 
+<GithubLink to="plugins/hydra_submitit_launcher">here</GithubLink>. Read this [doc](/docs/plugins/submitit_launcher) on installation/usage info.
+
+### Clean Install
+
+```commandline
+pip install hydra-fair-plugins  --pre --upgrade --upgrade-strategy=eager
+```
+
+### Upgrade from stable
+
+```commandline
+# Remove legacy fair internal submitit launcher plugin
+pip uninstall hydra-submitit -y
+pip install hydra-fair-plugins  --pre --upgrade --upgrade-strategy=eager
+```
+Check out [Hydra documentation](/docs/plugins/submitit_launcher) for  more info on ```Submitit``` launcher plugin.
+
+
+The depedency looks like 
+```commandline
+$ pip freeze | grep hydra
+hydra-core==1.0.0rc1
+hydra-fair-cluster==1.0.0rc1
+hydra-fair-plugins==1.0.0rc1
+hydra-fairtask==1.0.0rc1
+hydra-submitit-launcher==1.0.0rc3
+```
+
+Please refer to [Hydra upgrades](/docs/upgrades/0.11_to_1.0/config_path_changes) on what changes are needed for your app for upgrading to Hydra 1.0
+</details>
+
+<details>
+<summary>Downgrade From 1.0rc to stable</summary>
+
+Downgrade to stable in case you run into issues and need to be unblocked immediately.
+
+```commandline
+pip freeze | grep hydra | xargs pip uninstall -y
+pip install hydra-fair-plugins
+```
+</details>
+
+### Usage
+
+<details>
+ <summary>0.3.1 (stable)</summary>
+
+Once the plugins are installed, you can launch to the FAIR cluster by appending hydra/launcher=fairtask or hydra/launcher=submitit
+for example:
+
+```
+ python my_app.py -m hydra/launcher=submitit db=mysql,postgresql
+# or
+ python my_app.py -m hydra/launcher=fairtask db=mysql,postgresql
+```
+
+Both hydra-submitit and hydra-fairtask are providing sensible defaults for their configuration ([Submitit](https://github.com/fairinternal/hydra-fair-plugins/blob/master/plugins/hydra-submitit/hydra_plugins/submitit/conf/hydra/launcher/submitit.yaml), [fairtask](https://github.com/fairinternal/hydra-fair-plugins/blob/master/plugins/hydra-fairtask/hydra_plugins/fairtask/conf/hydra/launcher/fairtask.yaml))
+
+You can customize fairtask/submitit behavior much like you can customize anything else, from the command line or by overriding in your config file or composing in alternative launcher configuration.
+You can view the Hydra config (which includes the config for submitit or fairtask) with this command:
+```
+python my_app.py hydra/launcher=submitit --cfg=hydra
+```
+</details>
+
+
+<details>
+ <summary>1.0 (Release Candidate)</summary>
+
+
+For 1.0, ```fairtask``` usage remains the same. To use ```Submitit```, the command changes to:
+
+```commandline
+python my_app.py -m hydra/launcher=submitit_slurm db=mysql,postgresql
+```
+
+More info on ```Submitit``` launcher can be found [here](https://hydra.cc/docs/plugins/submitit_launcher)
+
+</details>

website/docs/fb/fbcode.md

@@ -0,0 +1,30 @@
+---
+id: fbcode
+title: Hydra at fbcode
+---
+
+## Differences in fbcode
+
+### Open source plugins
+#### Supported:
+ - hydra_ax_sweeper
+ - hydra_colorlog
+ - hydra_nevergrad_sweeper
+
+#### Unsupported:
+ - joblib launcher: Joblib's Loki backend does not work correctly when executed from a par file.
+
+### Facebook specified plugins
+ - fbcode_defaults : Changes configuration defaults to be appropriate for fbcode (e.g: Output directories are in `fbcode/outputs` and `fbcode/multirun`)
+ - configerator_config_source: See [doc](/fb/configerator-config-source.md)  for more information.
+
+#### TARGETS
+Hydra includes buck TARGETS you can use in fbcode. In general, if there is TARGET there are two options:
+1. You can depend on the TARGETS to use Hydra or a plugin.
+2. The TARGETS contains a runnable example.
+
+targets are under `github/facebookresearch/hydra`:
+- [:`hydra`](https://www.internalfb.com/intern/diffusion/FBS/browsedir/master/fbcode/github/facebookresearch/hydra) : Primary target to use in most cases. Includes `hydra_oss` and the `fbcode_defaults`.
+- `:hydra_oss` : Vanilla Hydra without any Facebook specific targets.
+- [`plugins`](https://www.internalfb.com/intern/diffusion/FBS/browsedir/master/fbcode/github/facebookresearch/hydra/plugins): Plugins that have a TARGETS file are runnable in fbcode.
+- [`examples`](https://www.internalfb.com/intern/diffusion/FBS/browsedir/master/fbcode/github/facebookresearch/hydra/examples): Examples that have a TARGETS file are runnable in `fbcode`. All tutorials (in [`examples/tutorials`](https://www.internalfb.com/intern/diffusion/FBS/browsedir/master/fbcode/github/facebookresearch/hydra/examples)) are supported. An example TARGET file can be found [here](https://www.internalfb.com/intern/diffusion/FBS/browsedir/master/fbcode/github/facebookresearch/hydra/examples/tutorials/basic/your_first_hydra_app/5_composition).

website/docs/fb/flow-launcher.md

@@ -0,0 +1,57 @@
+---
+id: flow-launcher
+title: Flow Launcher
+---
+
+The Flow Launcher plugin provides a way to launch application via `flow`.
+
+## Dependency
+To use the Flow Launcher, add the following to your `TARGETS`
+```commandline
+//github/facebookresearch/hydra/plugins/hydra_flow_launcher:hydra_flow_launcher
+```
+
+## Usage
+Add hydra/launcher=flow to your command line. Alternatively, override hydra/launcher in your config:
+```commandline
+defaults:
+  - hydra/launcher: flow
+```
+
+<details><summary>Discover Flow Launcher's config</summary>
+
+```yaml title="$ buck run @mode/opt  //path:my_app -- --cfg hydra -p hydra.launcher"
+
+# @package hydra.launcher
+_target_: hydra_plugins.flow_launcher_plugin.flow_launcher.FlowLauncher
+mode: flow
+owner: ${oc.env:USER}
+entitlement: gpu_pnb_fair
+pkg_version: fblearner.flow.canary:19e63cbf9945467281cf681bc8902c50
+driver_path: ''
+resource_requirements:
+  gpu: 0
+  cpu: 1
+  memory: 10g
+  region: null
+  capabilities: []
+  percent_cpu: null
+run_as_secure_group: fair_research_and_engineering
+retries: 2
+tags: []
+```
+</details>
+
+
+
+The Launcher currently support both `par` and `xar` style. You can override `resource_requirements` just like how you would via `flow-cli`.
+
+
+:::info NOTE
+Flow launcher only supports `@mode/opt`.
+:::
+
+To run the example application:
+```commandline
+buck run @mode/opt  //github/facebookresearch/hydra/plugins/hydra_flow_launcher/example:my_app -- --multirun
+```

website/docs/fb/intro.md

@@ -0,0 +1,28 @@
+---
+id: intro
+title: Hydra at Facebook
+---
+
+### Intro
+Facebook has multiple different environments, such as the **Internal FB Cluster**, the **FAIR Cluster** etc.
+
+The FB specific docs are describing the differences.
+
+### Release strategy
+Hydra's source of truth is the [GitHub repo](https://github.com/facebookresearch/hydra).
+
+Hydra is developed using release branches. Once a new major is released, it is maintained in patch only mode.
+Primary development is happening on the `master` branch.
+
+When a new major version of Hydra is released, a new release branch is created in Hydra repo. A corresponding Hydra version will be created inside `github/facebookresearch/hydra_VERSION` to track
+the release branch.
+
+Hydra is trying hard to remain backward compatible between two subsequent versions and in most cases the upgrade will be smooth.
+There could be some new deprecations warnings that should be fixed before the next major version.
+
+### Maintaining this documentation
+This documentation lives in in the Hydra repo which is publicly accessible. The pages will only normally render on the internal
+copy of the docs, but keep in mind that everyone can read those docs in the repo if they want to.
+
+1. Do not put anything sensitive here, no root passwords or launch codes.
+2. If you are in need to have sensitive Hydra related documentation please reach out to the maintainers of Hydra for help.