host_vars and group_vars in inventory directory are ignored by ansible-rulebook

[evonlanthen]

Please confirm the following

• I agree to follow this project's code of conduct.
• I have checked the current issues for duplicates.
• I understand that ansible-rulebook is open source software provided for free and that I might not receive a timely response.

Bug Summary

I have a simple ansible playbook which prints variables defined in host_vars and group_vars files defined in the inventory directory. If I execute the playbook with ansible-playbook (or ansible-runner), the tasks can be executed. If I trigger it by a webhook via ansible-rulebook the variables are not found.

Environment

OS: WSL2, Ubuntu 22.04.3 LTS
Ansible:

ansible [core 2.16.0]
  config file = None
  configured module search path = ['/home/evonlanthen/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/evonlanthen/.local/lib/python3.10/site-packages/ansible
  ansible collection location = /home/evonlanthen/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/evonlanthen/.local/bin/ansible
  python version = 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] (/usr/bin/python3)
  jinja version = 3.0.3
  libyaml = True

Ansible-rulebook:

1.0.6
  Executable location = /home/evonlanthen/.local/bin/ansible-rulebook
  Drools_jpy version = 0.3.9
  Java home = /usr/lib/jvm/java-17-openjdk-amd64
  Java version = 17.0.10
  Python version = 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]

Steps to reproduce

Directory structure:

.
├── inventories
│   └── production
│       ├── group_vars
│       │   └── mygroup.yml
│       ├── host_vars
│       │   └── localhost.yml
│       └── hosts.yml
├── playbook.yml
└── rulebook.yml

playbook.yml:

---
- hosts: mygroup
  gather_facts: false
  tasks:
    - name: Show vars
      ansible.builtin.debug:
        msg:
          - "groupvar: {{ mygroupvar }}"
          - "hostvar: {{ myhostvar }}"
      connection: local

rulebook.yml:

---
- name: Listen for selfservice events on a webhook
  hosts: all

  sources:
    - ansible.eda.webhook:
        host: 0.0.0.0
        port: 5000
      filters:
        - ansible.eda.insert_hosts_to_meta:
            host_path: payload.host
  rules:
    - name: Test rule
      condition: 1 == 1
      actions:
        - print_event:
            pretty: true
        - run_playbook:
            name: playbook.yml
            copy_files: true

inventories/production/hosts.yml:

---
mygroup:
  hosts:
    localhost

inventories/production/group_vars/mygroup.yml:

---
mygroupvar: 'hello from group var'

ansible-playbook command:

evonlanthen@SWP-HM60793 ~/ansible/rulebook $ ansible-playbook -i inventories/production/hosts.yml playbook.yml

PLAY [mygroup] *********************************************************************************************************************************************************************************************************************************************

TASK [Show vars] *******************************************************************************************************************************************************************************************************************************************
ok: [localhost] => {
    "msg": [
        "groupvar: hello from group var",
        "hostvar: hello from host var"
    ]
}

PLAY RECAP *************************************************************************************************************************************************************************************************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

ansible-rulebook command:

evonlanthen@SWP-HM60793 ~/ansible/rulebook $ ansible-rulebook -i inventories/production/hosts.yml -r rulebook.yml -v
2024-05-15 15:39:28,769 - ansible_rulebook.app - INFO - Starting sources
2024-05-15 15:39:28,769 - ansible_rulebook.app - INFO - Starting rules
2024-05-15 15:39:28,769 - drools.ruleset - INFO - Using jar: /home/evonlanthen/.local/lib/python3.10/site-packages/drools/jars/drools-ansible-rulebook-integration-runtime-1.0.6-SNAPSHOT.jar
2024-05-15 15:39:29 201 [main] INFO org.drools.ansible.rulebook.integration.api.rulesengine.AbstractRulesEvaluator - Start automatic pseudo clock with a tick every 100 milliseconds
2024-05-15 15:39:29,212 - ansible_rulebook.engine - INFO - load source ansible.eda.webhook
2024-05-15 15:39:29,505 - ansible_rulebook.engine - INFO - loading source filter ansible.eda.insert_hosts_to_meta
2024-05-15 15:39:29,505 - ansible_rulebook.engine - INFO - loading source filter eda.builtin.insert_meta_info
2024-05-15 15:39:29,786 - ansible_rulebook.engine - INFO - Waiting for all ruleset tasks to end
2024-05-15 15:39:29,786 - ansible_rulebook.rule_set_runner - INFO - Waiting for actions on events from Listen for selfservice events on a webhook
2024-05-15 15:39:29,786 - ansible_rulebook.rule_set_runner - INFO - Waiting for events, ruleset: Listen for selfservice events on a webhook
2024-05-15 15:39:29 786 [drools-async-evaluator-thread] INFO org.drools.ansible.rulebook.integration.api.io.RuleExecutorChannel - Async channel connected

webhook trigger:

evonlanthen@SWP-HM60793 ~ $ curl -H 'Content-Type: application/json' -d '{"host":"localhost"}' 127.0.0.1:5000/endpoint

ansible-rulebook output:

2024-05-15 15:41:27,647 - aiohttp.access - INFO - 127.0.0.1 [15/May/2024:14:41:27 +0100] "POST /endpoint HTTP/1.1" 200 159 "-" "curl/7.81.0"

** 2024-05-15 15:41:27.649608 [event] **********************************************************************************************************************************************************************************************************************
{'meta': {'endpoint': 'endpoint',
          'headers': {'Accept': '*/*',
                      'Content-Length': '20',
                      'Content-Type': 'application/json',
                      'Host': '127.0.0.1:5000',
                      'User-Agent': 'curl/7.81.0'},
          'hosts': ['localhost'],
          'received_at': '2024-05-15T13:41:27.647399Z',
          'source': {'name': 'ansible.eda.webhook',
                     'type': 'ansible.eda.webhook'},
          'uuid': '943699e9-eb36-4537-82f3-3df1e033c282'},
 'payload': {'host': 'localhost'}}
************************************************************************************************************************************************************************************************************************************************************

PLAY [mygroup] *****************************************************************

TASK [Show vars] ***************************************************************
fatal: [localhost]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'mygroupvar' is undefined. 'mygroupvar' is undefined\n\nThe error appears to be in '/tmp/eda131fp7ui/project/playbook.yml': line 5, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n  tasks:\n    - name: Show vars\n      ^ here\n"}

PLAY RECAP *********************************************************************
localhost                  : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0
2024-05-15 15:41:28,185 - ansible_rulebook.action.runner - INFO - Ansible runner Queue task cancelled
2024-05-15 15:41:28,185 - ansible_rulebook.action.run_playbook - INFO - Ansible runner rc: 2, status: failed
2024-05-15 15:41:28,185 - ansible_rulebook.action.run_playbook - ERROR -
PLAY [mygroup] *****************************************************************

TASK [Show vars] ***************************************************************
fatal: [localhost]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'mygroupvar' is undefined. 'mygroupvar' is undefined\n\nThe error appears to be in '/tmp/eda131fp7ui/project/playbook.yml': line 5, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n  tasks:\n    - name: Show vars\n      ^ here\n"}

PLAY RECAP *********************************************************************
localhost                  : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0

Actual results

Ansible-playbook run fails with undefined variable (it doesn't matter if a variable from group_vars or hosts_vars is used first).

Expected results

Successful ansible playbook run.

Additional information

No response

[evonlanthen]

Synchronizing the directories to the ansible playbook main directory solves the issue:

for TYPE in host group; do
  rsync -av inventories/*/${TYPE}_vars/ ${TYPE}_vars/
done

It there a better way?

[Alex-Izquierdo]

Hi @evonlanthen Thanks for the report.
Indeed I see two different bugs.

1. When the inventory is a file, we only copy the file and we don't take into account other potential files.
2. If the inventory is a directory, we are not setting correctly the path.

I'm going to fix them in #683

[evonlanthen]

Hi @Alex-Izquierdo

Thanks for your investigation and your fix.

Kind regards
Elmar

[mkanoor]

@evonlanthen When you ran the ansible-rulebook command line can you use the directory name for inventory without the file name e.g

ansible-rulebook -i inventories

[evonlanthen]

@evonlanthen When you ran the ansible-rulebook command line can you use the directory name for inventory without the file name e.g

ansible-rulebook -i inventories

If I specify the directory only, no hosts are found anymore. The same happens with:

ansible-rulebook -i inventories/production

[Alex-Izquierdo]

@evonlanthen When a directory is set it should take the whole directory as inventory. This is what is going to fix: #684