Problems in expansion of a setcounter macro (Use of \csname doesn't match its definition)?

The problem is the line

\expandafter\protected@edef\expandafter\csname\tcsa\endcsname{\expandafter\csname\tcsb\endcsname}% 

Here, you are expanding \tcsa (which you don't actually need to do), but not expanding \csname before \protected@edef. The result is you redefine \csname: a Bad Thing! You need instead

\expandafter\protected@edef\csname\tcsa\endcsname{\csname\tcsb\endcsname}% 

which expands the name first, then does \protected@edef with the correct name. I've removed another unneeded (but harmless) \expandafter in the definition.

The error you get

Use of \csname doesn't match its definition 

is a clear sign that something has gone wrong in a definition, because it means you have redefined \csname. This happens in

\expandafter\protected@edef\expandafter\csname\tcsa\endcsname{% \expandafter\csname\tcsb\endcsname} 

because the first \expandafter expands the second wich expands \tcsa and then you end with

\protected@edef\csname<expansion of \tcsa\endcsname{% \expandafter\csname\tcsb\endcsname} 

which will indeed redefine \csname with a very peculiar parameter text.

This is cured, as Joseph Wright points out, by omitting the second \expandafter (and also the third, which really does nothing useful):

\expandafter\protected@edef\csname\tcsa\endcsname{\csname\tcsb\endcsname} 

No other \expandafter is needed, just the first one in order to present \protected@edef with a symbolic token.

However, there are other problems in your code.

1. \ifx#1\undefined\fi is completely wrong; if you want to test for emptyness of the argument, use \if\relax\detokenize{#1}\relax

2. Several of the definitions you make are useless and, as Joseph Wright remarks, expansion is done in the wrong order.

Finally, the solution is far simpler. Only one “parent subcounter” is necessary, unless you want to be able to nest subcounters. Also, \setcounter and \value already do expansion of their argument.

\documentclass{article} \usepackage{amsmath} \usepackage{listings} \usepackage[colorlinks]{hyperref} % modded from {subequations}, `kpsewhich amsmath.sty` \newcounter{parent@@subcounter}% Counter for the parent subcounter \makeatletter \newif\if@subanyempty \newenvironment{subcounter}[1][]{% \if\relax\detokenize{#1}\relax \@subanyemptytrue \typeout{subcounter: Didn't get argument: '#1'; skipping start}% \else \def\subany@argument{#1}% for the end part \@subanyemptyfalse % actually redundant \refstepcounter{#1}% step the given counter \protected@edef\theparent@@subcounter{\csname the#1\endcsname}% \setcounter{parent@@subcounter}{\value{#1}}% \setcounter{#1}{0}% \ifcsname theH#1\endcsname % for hyperref \@namedef{theH#1}{\theparent@@subcounter\alph{#1}}% \fi \@namedef{the#1}{\theparent@@subcounter\alph{#1}}% \fi \ignorespaces }{% \if@subanyempty \typeout{subanything: skipping end}% \else \setcounter{\subany@argument}{\value{parent@@subcounter}}% \fi \ignorespacesafterend } \makeatother \begin{document} Hello... \begin{subequations}\label{eq:ex1} \begin{minipage}{0.45\textwidth} \begin{align} \label{eq:ex1a} a = b + c \end{align} \end{minipage} \begin{minipage}{0.45\textwidth} \begin{align} \label{eq:ex1b} x = y + z \end{align} \end{minipage} \end{subequations} \vspace{\baselineskip} Ref'ing: master eq.~\ref{eq:ex1}; inner a eq.~\ref{eq:ex1a}, inner b eq.~\ref{eq:ex1b} ... \begin{subcounter}[lstlisting] \label{lst:ex1} \noindent\begin{minipage}[t]{.325\textwidth} \begin{lstlisting}[basicstyle=\scriptsize\ttfamily, caption={[short] Some instructions here; the font here is \texttt{\ttdefault}.}, escapechar=!, showlines=true, label=lst:ex1a, columns=fixed, frame=tlrb] 080484c4 <list>: 80484c4: cmd one 80484c7: cmd two 80484ca: cmd three, four 80484cf: cmd five 80484d6: cmd six, seven 80484dd: cmd more than enough 80484e0: cmd not_even_joking \end{lstlisting} \end{minipage} \hspace{1cm} \noindent\begin{minipage}[t]{.325\textwidth} \begin{lstlisting}[basicstyle=\scriptsize\ttfamily, caption={[short] Some instructions here; the font here is \texttt{\ttdefault}.}, escapechar=!, showlines=true, label=lst:ex1b, columns=fullflexible, % basewidth=\tlen, frame=tlrb] 080484c4 <list>: 80484c4: cmd one 80484c7: cmd two 80484ca: cmd three, four 80484cf: cmd five 80484d6: cmd six, seven 80484dd: cmd more than enough 80484e0: cmd not_even_joking \end{lstlisting} \end{minipage} \end{subcounter} Ref'ing: inner a listing~\ref{lst:ex1a}, inner b listing~\ref{lst:ex1b}, master \ref{lst:ex1} \end{document} 

Here's an expl3 version that allows nesting, by defining a “parent” counter the first time the environment is used with a particular counter.

\documentclass{article} \usepackage{amsmath} \usepackage{listings} \usepackage{xparse} \usepackage[colorlinks]{hyperref} \ExplSyntaxOn \NewDocumentEnvironment{subcounter}{o} { \IfValueT{#1}{ \sdaau_setup_subcounter:n { #1 } } \ignorespaces } { \IfValueT{#1}{ \sdaau_restore_subcounter:n { #1 } } \ignorespacesafterend } % an expl3 version of \protected@edef \cs_set_eq:Nc \sdaau_protected_edef:Npn { protected@edef } \cs_generate_variant:Nn \sdaau_protected_edef:Npn { c } \cs_new_protected:Npn \sdaau_setup_subcounter:n #1 { \cs_if_exist:cF { c@parent@@#1 } {% for the first time a counter is subbed \newcounter{ parent@@#1 } } \refstepcounter{ #1 } \sdaau_protected_edef:cpn { theparent@@#1 } { \use:c { the#1 } } \setcounter{ parent@@#1 } { \value { #1 } } \setcounter{ #1 } { 0 } \cs_set:cpn { the#1 }{ \use:c { theparent@@#1 } \alph{#1} } \cs_if_exist:cT { theH#1 } { \cs_set_eq:cc { theH#1 } { the#1 } } } \cs_new_protected:Npn \sdaau_restore_subcounter:n #1 { \setcounter{#1}{\value{parent@@#1}} } \ExplSyntaxOff \begin{document} Hello... \begin{subcounter}[equation]\label{eq:ex1} \begin{minipage}{0.45\textwidth} \begin{align} \label{eq:ex1a} a = b + c \end{align} \end{minipage} \begin{minipage}{0.45\textwidth} \begin{align} \label{eq:ex1b} x = y + z \end{align} \end{minipage} \end{subcounter} \vspace{\baselineskip} Ref'ing: master eq.~\ref{eq:ex1}; inner a eq.~\ref{eq:ex1a}, inner b eq.~\ref{eq:ex1b} ... \begin{subcounter}[lstlisting] \label{lst:ex1} %\begin{subcounter}% switch the comments to test \noindent\begin{minipage}[t]{.325\textwidth} \begin{lstlisting}[basicstyle=\scriptsize\ttfamily, caption={[short] Some instructions here; the font here is \texttt{\ttdefault}.}, escapechar=!, showlines=true, label=lst:ex1a, columns=fixed, frame=tlrb] 080484c4 <list>: 80484c4: cmd one 80484c7: cmd two 80484ca: cmd three, four 80484cf: cmd five 80484d6: cmd six, seven 80484dd: cmd more than enough 80484e0: cmd not_even_joking \end{lstlisting} \end{minipage} \hspace{1cm} \noindent\begin{minipage}[t]{.325\textwidth} \begin{lstlisting}[basicstyle=\scriptsize\ttfamily, caption={[short] Some instructions here; the font here is \texttt{\ttdefault}.}, escapechar=!, showlines=true, label=lst:ex1b, columns=fullflexible, % basewidth=\tlen, frame=tlrb] 080484c4 <list>: 80484c4: cmd one 80484c7: cmd two 80484ca: cmd three, four 80484cf: cmd five 80484d6: cmd six, seven 80484dd: cmd more than enough 80484e0: cmd not_even_joking \end{lstlisting} \end{minipage} \end{subcounter} Ref'ing: inner a listing~\ref{lst:ex1a}, inner b listing~\ref{lst:ex1b}, master \ref{lst:ex1} \end{document}