Full Disclosure:

I am writing this question to then answer it myself. I searched the internet for a full day at work and was not able to find a solution that worked for me. I even compiled my own GoogleAuthenticator PAM module to add more logging. Not even running strace on the OpenVPN process and its children led me to a solution.

Use Case

• Launch VPN using OpenVPN in an EC2
• Use PAM GoogleAuthenticator Module
• OS: CentOS

Setup

• login to the EC2
• create a client
• add MFA token to user (client) using the provided token generator while also saving this token to the proper location for the PAM module to detect it
  • create the <user>.ovpn file for this user

The below script will create a Linux user, and then create an MFA secret saved to the location specified in the PAM config, note the permissions 600, MFA_USER is a pre-created user that I created named gauth

function generate_mfa() { user_id=$1 if [ "$user_id" == "" ]; then echo "ERROR: No user id provided to generate MFA token" >&2 exit 1 fi echo "INFO: Creating user ${user_id}" >&2 useradd -s /bin/nologin "$user_id" echo "> Please provide a password for the user" >&2 passwd "$user_id" echo "INFO: Generating MFA Token" >&2 google-authenticator -t -d -r3 -R30 -f -l "${MFA_LABEL}" -s "${MFA_DIR}/${user_id}" chown "${MFA_USER}:${MFA_USER}" "$MFA_DIR/${user_id}" chmod 600 "${MFA_DIR}/${user_id}" } 

PAM Config for OpenVPN

auth required /usr/lib64/security/pam_google_authenticator.so secret=/etc/openvpn/google-authenticator/${USER} user=gauth forward_pass auth include system-auth use_first_pass account include system-auth use_first_pass password include system-auth use_first_pass session include system-auth use_first_pass auth required pam_deny.so 

Issue

• Using Tunnelblick configured with my client.ovpn, I am then prompted to log in with my username and password.
  • The format of password is inline: <password><MFA_TOKEN>, this is stripped out with the forward_pass directive
• I enter in my proper credentials and am always met with unauthorized

Logs

• To check out my issue I logged onto the VPN instance via ssh and inspected my PAM/auth logs tail /var/log/secure

Sep 10 22:33:43 ip-OMITTED openvpn(pam_google_authenticator)[12862]: Accepted google_authenticator for ryan Sep 10 22:33:43 ip-OMITTED openvpn(pam_google_authenticator)[12862]: Failed to update secret file "/etc/openvpn/google-authenticator/ryan": Permission denied 

Aha! "Permission Denied"

So then check my permissions:

[root@ip-OMITTED centos]# ls -lah /etc/openvpn/google-authenticator/ drwxr-xr-x. gauth gauth . drwxr-xr-x. root root .. -rw-------. gauth gauth ryan 

• Hmm, these permissions 600 seem right. The directories are executable and I am using the gauth user in my PAM config.

What on earth could be wrong with my configuration?

• the gauth user exists :check:
• the permissions are right :check: