2

I am trying to look up the IPv4 address (a single one, which I will assert on!) for a given name using Ansible and use that as a fact "down the line".

An MWE looks like this:

--- # vim: set autoindent smartindent tabstop=2 shiftwidth=2 softtabstop=2 expandtab filetype=ansible: - name: Preparations hosts: localhost connection: local vars: host2lookup: "example.com" tasks: - getent: database: ahostsv4 key: "{{host2lookup}}" - debug: var: getent_ahostsv4 - name: Lookup for {{host2lookup}} debug: msg: "{{lookup('dig', '{{host2lookup}}./A')}}"

... and fails as follows (relevant excerpt):

 localhost failed | msg: An unhandled exception occurred while running the lookup plugin 'dig'. Error was a <class 'ansible.errors.AnsibleError'>, original message: The dig lookup requires the python 'dnspython' library and it is not installed

So I thought to myself I'd install the dnspython library for the appropriate Python version, making the playbook look as follows:

--- # vim: set autoindent smartindent tabstop=2 shiftwidth=2 softtabstop=2 expandtab filetype=ansible: - name: Preparing Ansible host hosts: localhost connection: local tasks: - name: Install dnspython package on Ansible host block: - name: Installing python-dnspython package apt: pkg: - python-dnspython update_cache: yes install_recommends: no state: present when: ansible_python.version.major == 2 - name: Installing python3-dnspython package apt: pkg: - python3-dnspython update_cache: yes install_recommends: no state: present when: ansible_python.version.major == 3 become: true - name: Preparations hosts: localhost connection: local vars: host2lookup: "example.com" tasks: - getent: database: ahostsv4 key: "{{host2lookup}}" - debug: var: getent_ahostsv4 - name: Lookup for {{host2lookup}} debug: msg: "{{lookup('dig', '{{host2lookup}}./A')}}"

But the ultimate failure remains the same. Either that error is misleading or something else is fishy ...

Executing playbook dbg.yml - Preparing Ansible host on hosts: localhost - Gathering Facts... localhost ok Installing python-dnspython package... Installing python3-dnspython package... localhost ok - Preparations on hosts: localhost - Gathering Facts... localhost ok getent... localhost ok debug... localhost ok: { "changed": false, "getent_ahostsv4": { "93.184.216.34": [ "RAW" ] } } Lookup for example.com... localhost failed | msg: An unhandled exception occurred while running the lookup plugin 'dig'. Error was a <class 'ansible.errors.AnsibleError'>, original message: The dig lookup requires the python 'dnspython' library and it is not installed - Play recap - localhost : ok=5 changed=0 unreachable=0 failed=1 rescued=0 ignored=0

What other than the appropriate dnspython package am I missing here?

The dnsutils package (containing dig) and its library dependencies is installed. Adding these as prerequisites into the apt: task therefore makes no difference.

Found another smoking gun, though:

$ python3 -m dnspython /usr/bin/python3: No module named dnspython $ sudo -H pip3 install dnspython Requirement already satisfied: dnspython in /usr/lib/python3/dist-packages

I stand corrected. As per the examples page I should import something like dns.query (python3 -m dns.query). And that succeeds.

My ansible.cfg is this:

$ cat ansible.cfg [defaults] nocows = true command_warnings = false stdout_callback = unixy display_skipped_hosts = false interpreter_python=auto

The whole thing runs on Ubuntu 18.04.

As per request in a comment, I am providing more output. This is still on Ubuntu 18.04. With the (above shown) ansible.cfg in place, I get the following output:

$ ansible localhost -m setup -a 'filter=ansible_python*' localhost | SUCCESS => { "ansible_facts": { "ansible_python": { "executable": "/usr/bin/python2", "has_sslcontext": true, "type": "CPython", "version": { "major": 2, "micro": 17, "minor": 7, "releaselevel": "final", "serial": 0 }, "version_info": [ 2, 7, 17, "final", 0 ] }, "ansible_python_version": "2.7.17" }, "changed": false }

However, ansible-playbook behaves differently here. Consider the following playbook:

--- - hosts: all tasks: - debug: var: ansible_python - debug: var: ansible_python_version

The output becomes:

$ ansible-playbook -i localhost, -c local dbg.yml Executing playbook dbg.yml - all on hosts: all - Gathering Facts... localhost ok debug... localhost ok: { "ansible_python": { "executable": "/usr/bin/python3", "has_sslcontext": true, "type": "cpython", "version": { "major": 3, "micro": 9, "minor": 6, "releaselevel": "final", "serial": 0 }, "version_info": [ 3, 6, 9, "final", 0 ] }, "changed": false } debug... localhost ok: { "ansible_python_version": "3.6.9", "changed": false } - Play recap - localhost : ok=3 changed=0 unreachable=0 failed=0 rescued=0 ignored=0

... comment out the following line from ansible.cfg:

interpreter_python=auto

... and the output (when rerunning the previous command) becomes:

$ ansible-playbook -i localhost, -c local dbg.yml Executing playbook dbg.yml - all on hosts: all - Gathering Facts... localhost ok debug... localhost ok: { "ansible_python": { "executable": "/usr/bin/python", "has_sslcontext": true, "type": "CPython", "version": { "major": 2, "micro": 17, "minor": 7, "releaselevel": "final", "serial": 0 }, "version_info": [ 2, 7, 17, "final", 0 ] }, "changed": false } debug... localhost ok: { "ansible_python_version": "2.7.17", "changed": false } - Play recap - localhost : ok=3 changed=0 unreachable=0 failed=0 rescued=0 ignored=0

Nothing in the documentation suggests that this should happen.

Improve this question
5
  • Update the question and add the output of the command ansible localhost -m setup | grep ansible_python_version Commented Dec 18, 2019 at 13:33
  • @VladimirBotka I honestly appreciate your effort, but what does the ansible command line tool have to do with behavior I see with ansible-playbook? I will amend the question shortly. You'll notice that the two tools behave differently in this exact case. No clue why, but it can be reproduced. Commented Dec 18, 2019 at 13:42
  • @VladimirBotka amended the question with requested information ... and a little more, since the information you requested isn't representative of the issue I am encountering. Commented Dec 18, 2019 at 13:58
  • You did not amend the question with the requested information. I asked you to use -hosts: localhost, but you used - hosts: all. This is really little more. Commented Dec 18, 2019 at 14:15
  • @VladimirBotka Huh? Which part of ansible localhost -m setup -a 'filter=ansible_python*' (other than using a filter instead of grep) lacks the requested information? Your request was for output from ansible and I pointed out the discrepancy with that of ansible-playbook based on the interpreter_python=auto setting. Commented Dec 18, 2019 at 14:23

3 Answers 3

Reset to default
2

See:

See also Setting the Python interpreter that explains various configurations' options, variables, and debugging.

Q: "None of that explains why - and that is/was my point - ansible_python wrongly claims to be running on Python 3.x on the local node. Because all I do by setting interpreter_python in ansible.cfg is to give Ansible free rein on how it interprets things."

Short answer: You can't configure the version of Python the Ansible utils use ($ ansible --version). INTERPRETER_PYTHON configures the Python version used by the Ansible modules on the remote hosts.

A: Configuration option INTERPRETER_PYTHON works as expected. There is nothing wrong when Python3 is discovered when asked by interpreter_python=auto.

See implementation details of the function discover_interpreter. When the version of the distro is not in _INTERPRETER_PYTHON_DISTRO_MAP

"return nearest previous version we're newer than"

def _version_fuzzy_match(version, version_map): ... # slot match; return nearest previous version we're newer than kpos = bisect.bisect(sorted_looseversions, find_looseversion) 
interpreter_python=auto 
shell> ansible localhost -m setup -a 'filter=ansible_python_version' localhost | SUCCESS => { "ansible_facts": { "ansible_python_version": "3.7.3", "discovered_interpreter_python": "/usr/bin/python3" }, "changed": false }

"The default value of auto_legacy provides all the same behavior, but for backward compatibility with older Ansible releases that always defaulted to /usr/bin/python, will use that interpreter if present (and issue a warning that the default behavior will change to that of auto in a future Ansible release."

interpreter_python=auto_legacy 
shell> ansible localhost -m setup -a 'filter=ansible_python_version' [DEPRECATION WARNING]: Distribution Ubuntu 19.04 on host localhost should use /usr/bin/python3, but is using /usr/bin/python for backward compatibility with prior Ansible releases. A future Ansible release will default to using the discovered platform python for this host. See https://docs.ansible.com/ansible/2.9/reference_appendices/interpreter_discovery.html for more information. This feature will be removed in version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg. localhost | SUCCESS => { "ansible_facts": { "ansible_python_version": "2.7.16", "discovered_interpreter_python": "/usr/bin/python" }, "changed": false }

See the notes and examples below.

Q: "What does the ansible command line tool have to do with the behavior I see with ansible-playbook?"

A: Both ansible command line tool and ansible-playbook will report the same value of ansible_python_version. For example,

shell> ansible localhost -m setup | grep ansible_python_version "ansible_python_version": "2.7.16", 
shell> cat playbook.yml - hosts: localhost tasks: - debug: var: ansible_python_version 
shell> ansible-playbook playbook.yml ok: [localhost] => { "ansible_python_version": "2.7.16" }
Notes
  1. INTERPRETER_PYTHON configuration parameter says:

Path to the Python interpreter to be used for module execution on remote targets

See How to choose a python interpreter for Ansible playbook?

  1. Ansible will tell what Python is used on master
shell> ansible --version ansible 2.9.2 config file = /home/vlado/.ansible.cfg configured module search path = [u'/home/vlado/.ansible/my_modules'] ansible python module location = /usr/lib/python2.7/dist-packages/ansible executable location = /usr/bin/ansible python version = 2.7.16 (default, Oct 7 2019, 17:36:04) [GCC 8.3.0]
  1. dig says

The below requirements are needed on the local master node that executes this lookup: dnspython

  1. It's possible to install both versions
shell> dpkg -l | grep dnspython ii python-dnspython 1.16.0-1 all DNS toolkit for Python ii python3-dnspython 1.16.0-1 all DNS toolkit for Python 3

  1. See Ansible packages in Ubuntu what version of Python was used.

  2. ansible_python_interpreter works as expected

On master(Ubuntu) the playbook

shell> cat playbook.yml - hosts: test_01 tasks: - getent: database: hosts 
shell> cat hosts test_01 ansible_python_interpreter=/usr/local/bin/python3.6 
shell> ansible-playbook playbook.yml -i hosts -vvv | grep python

gives

ansible python module location = /usr/lib/python2.7/dist-packages/ansible python version = 2.7.16 (default, Oct 7 2019, 17:36:04) [GCC 8.3.0]

...

<test_01> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="admin"' -o ConnectTimeout=10 -o ControlPath=/export/home/vlado.config/.ansible/cp/5a3ab05cf7 -tt test_01 '/bin/sh -c '"'"'sudo -H -S -n -u root /bin/sh -c '"'"'"'"'"'"'"'"'echo BECOME-SUCCESS-xnvwkdksdqqoehvrgebdgcnrzcrmoszv ; /usr/local/bin/python3.6 /home/admin/.ansible/tmp/ansible-tmp-1576667662.62-67468482045094/AnsiballZ_getent.py'"'"'"'"'"'"'"'"' && sleep 0'"'"''

  • ansible-playbook on master runs with python version = 2.7.16
  • module on remote (test_01) runs with /usr/local/bin/python3.6

Change ansible_python_interpreter

shell> cat hosts test_01 ansible_python_interpreter=/usr/local/bin/python2.7

the same playbook gives

ansible python module location = /usr/lib/python2.7/dist-packages/ansible python version = 2.7.16 (default, Oct 7 2019, 17:36:04) [GCC 8.3.0]

...

<test_01> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="admin"' -o ConnectTimeout=10 -o ControlPath=/export/home/vlado.config/.ansible/cp/5a3ab05cf7 -tt test_01 '/bin/sh -c '"'"'sudo -H -S -n -u root /bin/sh -c '"'"'"'"'"'"'"'"'echo BECOME-SUCCESS-fidpgwptggdoizirumvjmzuorxqawypu ; /usr/local/bin/python2.7 /home/admin/.ansible/tmp/ansible-tmp-1576668475.22-210929624017605/AnsiballZ_getent.py'"'"'"'"'"'"'"'"' && sleep 0'"'"''

  • ansible-playbook on master runs with python version = 2.7.16
  • but module on remote (test_01) runs with /usr/local/bin/python2.7 as requested
Improve this answer
8
  • Yes, but none of that explains why - and that is/was my point - ansible_python wrongly claims to be running on Python 3.x on the local node. Because all I do by setting interpreter_python in ansible.cfg is to give Ansible free rein on how it interprets things. But since I am depending on its choices and its information being consistent (which it evidently isn't), things blow up in my face. Regarding point 4, it's completely beside the point, because I am attempting to install the appropriate version based on ansible_python, but Ansible simply lies to me ... Commented Dec 18, 2019 at 9:01
  • There is no evidence that ansible_python wrongly claims to be running on Python 3.x on the local node. I've added more details that might help to clarify the situation. Commented Dec 18, 2019 at 11:51
  • The evidence is the behavior one gets with when: ... the wrong package that gets installed (provided neither, i.e. python-dnspython/python3-dnspython, was installed before). This is actually reproducible. But you're right, this only happens with interpreter_python=auto set in ansible.cfg for me. But then I consistently see it (and I have actually used the debug module to verify the value of ansible_python is wrong). Only if python-dnspython is preinstalled (samba installs it, for example) don't I see the issue. Otherwise I can reproduce on all 18.04 so far. Commented Dec 18, 2019 at 13:19
  • Just to make sure. I understand that the lookup plugins get called on the control/master. In my real-world case the Preparing Ansible host part is getting actually executed on the control/master, but the Preparations part gets executed on the remote machine(s). With the sole exception of the loopkup plugin, so I understand that the dnspython package needs to be installed on control/master and that the result of the lookup is passed (by value) to the remote machine(s). In my MWE I had to condense this for obvious reasons. Commented Dec 18, 2019 at 13:33
  • A: Both ansible command line tool and ansible-playbook will report the same value of ansible_python_version. ... unfortunately not true, at least not unconditionally (see amendments to the question). Commented Dec 18, 2019 at 14:12
1

Since this confused the hell out of me and cost me about an hour to figure out, I am going to post an answer.

As you can see I am using ansible_python to determine the Python version and install the appropriate version of dnspython. Since this is on Ubuntu 18.04 it defaults to python3. I am suppressing the respective warning with interpreter_python=auto in the ansible.cfg inside the same folder.

But here's the bummer. Apparently connection: local somehow violates the rule about using python3 whose dnspython package is installed. It made sense once I looked for dig.py (hoping for the Ansible lookup module of the same name). And indeed I found it:

$ locate dig.py /usr/lib/python2.7/dist-packages/ansible/plugins/lookup/dig.py /usr/lib/python2.7/dist-packages/ansible/plugins/lookup/dig.pyc

... alas, this means my version of Ansible uses Python 2.7 and so the dig module won't be able to make use of the python3-dnspython (Debian) package which I installed.

Still puzzling. But with the cause determined, I can work around it now.

Improve this answer
0

In ansible, lookup's are executed locally where ansible runs, not on the target server.

So you will have to ensure that dnspython is installed on your ansible machine.

Improve this answer

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.