2

I've been messing with TeX primitives, particularly catcodes, to handle diverse LaTeX situations (and to extend my wings).

I created a token gobbler with the help of https://tex.stackexchange.com/a/33536/13552

Why can't I see any 10s in my list of category codes?

Would it be safe to test for \@empty and if true, use \g@addto@macro{\tempa}{\fbox{ }}%

\documentclass{article} \begin{document} \makeatletter \def\scanfunction#1{#1}% Don't know why this is here \let\tempa\@empty \def\scan@letters#1#2{% https://tex.stackexchange.com/a/33536/13552 \bgroup\catcode`\ \active \let\ \fbox{ }\egroup%Try changing space to visible box \def\getcategory##1{\the\catcode`##1}% ` changes char into ASCI number required for \catcode \def\currcatcode{\getcategory{#1}}% put everything into unified token for \if \currcatcode, % spurious space intentional % first character missing \if 10\currcatcode% \fbox{#2}% if catcode 10 fed, print it as a box \fi \g@addto@macro{\tempa}{#1\hskip 0pt plus 1sp minus 1sp}% \ifx#2\@empty%\@empty is what the end of the line will look like ^^M? \else \expandafter\scan@letters % recall yourself using #2 until this conditional becomes true \fi #2} \def\scan#1{% %\catcode"0020 11 % Tried changing space to catcode 11 %\the\catcode"0020 % Test to ensure change happened \scan@letters #1\@empty } \makeatother \scan{ mac::exception == } % \scan{ } % crashes with "Improper alphabetic constant." \begin{itemize} \item \tempa \end{itemize} \end{document}

Of course, I expect the following from \scan{ mac::exception == }

10, 10, 11, 11, 11, 12, 12, 11, 11, 11, 11, 11, 11, 11, 11, 11, 10, 10, 12, 12, 10, 10,

enter image description here

My Favorites for Reference

Obviously, it is possible to see 10.

\the\catcode`\

result: 13 (mistakenly expected 10)

\the\catcode`\ %

result: 10

\the\catcode"0020

result: 10

24.06.2016 Update with Practical Solution

This is perhaps useful to know, so I'm putting it here. Using the context and David Carlisle's answer to understand \afterassignment so that I could test for category 10 (space), I modified my code to add glue after every non-space character such that the LaTeX line-breaking mechanism would trigger when

  • the total width of all character boxes = \textwidth

Sample

\documentclass{article} \usepackage{fontspec} \newfontfamily\macmonofont{Inconsolatazi4-Regular.otf} %\makeatletter % or \catcode"0040=11 \def\macmono#1{\xscan#1\relax}% calls xscan which looks ahead one token, #1 \def\xscan{\afterassignment\xxscan\let\token= }% assign single token to \token and call \xxscan \def\xxscan{% \macmonofont% apply mono font \ifx\token\relax\normalfont\else%test for end-of-line or end of group and switch back to normal font \ifcat\token\space% \token% token is catcode 10 \spaceskip=.5em% remove glue from space for fixed-width space \xspaceskip=.5em% remove glue from space for fixed-width space \else% \token\hskip 0pt plus 1sp minus 1sp % add glue to any non-catcode 10 (space) \fi \expandafter\xscan \fi} %\makeatother % or \catcode"0040=12 \parindent=0pt % remove firstpar autoindent \obeylines% insert \par after each end-of-line (^^M) \begin{document} \begin{itemize} \item Some random error created by C++: \macmono{mac::IOException [File: sharecpp/trunk/PROJECTS/SpecialOps/B/Build/Implementation/Library/GenerateDocumentation.cpp, Line: 999] Found documentation overview begin marker more than once in DocumentationManual! mac::IOException [File: sharecpp/trunk/PROJECTS/SpecialOps/B/Build/Implementation/Library/DocumentationThread.cpp, Line: 934] Processing of DocumentationManual failed while generating DocumentationOverview!} and some trailing test to make sure font changing works. \end{itemize} \end{document}

Output

enter image description here

Improve this question

1 Answer 1

Reset to default
6

An unbraced catcode 10 character is never taken as an undelimited argument so that you can go

\def\foo#1#2#3{...} \foo a b c

and have a, b and c (rather than the spaces between them) be the three arguments.

The usual solution is to use \futurelet (or as below \afterassignment and \let) but it depends what you want to do.

Your catcode changes such as 

\catcode`\ \active

have no effect as the spaces have already been tokenized in the argument to your \scan macro.

10, 10,

Also you should not expect consecutive 10 even if you fixed the scanner as any number of consecutive spaces in the input file are tokenized to a single catcode 10 token

enter image description here

\documentclass{article} \def\scan#1{\xscan#1\relax} \def\xscan{\afterassignment\xxscan\let\tmp= } \def\xxscan{% \ifx\tmp\relax\else \ifcat\tmp\space10 \else \ifcat\tmp a11 \else \ifcat\tmp 112 \else%... \fi\fi\fi \expandafter\xscan \fi} \begin{document} \scan{ mac::exception == } \end{document}
Improve this answer
8
  • Ah, that helps. What I wanted to do is add glue to each character, except I noticed that the spaces get gobbled up (or the existing glue expands as far as possible before line breaking effectively removing visible interword spaces). I wanted to define the space to have a fixed width to ensure they exist even when allowing LaTeX to line break anywhere. First I figured I should understand why my 10s are not showing up. Commented Jun 23, 2016 at 16:09
  • @macmadness86 See I added some code, but for the use case in your comment, don't do any of this! just set \spaceskip and \xspaceskip to fixed lengths. Commented Jun 23, 2016 at 16:11
  • Haha, well it was fun doing this experiment anyway. I tried doing \spaceskip=10pt \xspaceskip=10pt for an obvious effect and in seemed to have not effect. I am trying to be mindful of scopes. Also, if your scanner encounters any of the other catcodes, they won't show up I think. Commented Jun 23, 2016 at 16:27
  • Duh, it is not working because I am eating up all of the spaces in my original code. I should probably fix that first before complaining :) It is just hard to figure out because the only thing similar to my original code is \scan{ mac::exception == } Commented Jun 23, 2016 at 16:32
  • 1
    @macmadness86 with let you can also detect { and } (catcode 1 and 2) which you can not get with a macro argument, by the time you have used the above to get catcode 1,2,10, you may as well use it for all of them:-) Commented Jun 24, 2016 at 8:13

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.