In Japan, modified versions of TeX and LaTeX called ptex and platex are sometimes used. TeXShop has been modified by Makoto Inoue, Seiji Zenitani, Mitsuhiro Shishikura, and Yusuke Terada to deal with these versions. Their modifications are activated by a small number of items in the TeXShop Preferences Dialog.

In recent years, Yusuke Terada has become a particularly close collaborator, and many recent TeXShop additions are due to him. Recently a number of questions were asked about the Preference Items for Japan. Below are these questions and Yusuke Terada's answers.

QUESTION: In the Misc prefs, there is a section "pTeX support" with checkbox "utf export". What does it do?

ANSWER: In Japan, many kinds of TeX engines are used, such as pTeX, upTeX, XeTeX and LuaTeX. pTeX is the most traditional engine. Though upTeX is its successor, pTeX is still the most widely used engine in Japan.

pTeX does not support Unicode fully, while upTeX can handle Unicode appropriately. Though pTeX accepts TeX sources written in UTF-8, it converts internally the sources to EUC-JP. EUC-JP is one of the traditional Japanese encodings, which has much smaller character set than Unicode. So there are Unicode characters which can't be converted into EUC-JP. You can handle such Unicode characters with pTeX by using otf.sty.

For example, if you compile the following UTF-8 source with pLaTeX, the character "鷗" is lost because it can't be converted into EUC-JP. \documentclass{jsarticle} \begin{document} 森鷗外 was a famous novelist in Japan. \end{document}

Instead, you need to write the source as follows.

    \documentclass{jsarticle}
    \usepackage{otf}
    \begin{document}
    森\UTF{9DD7}外 was a famous novelist in Japan.
    \end{document}

However, it is too difficult for normal users to input sources in this way. The TeXShop feature "utf export" helps this. When this option is enabled, "鷗" is converted to "\UTF{9DD7}" when the document is saved, and it is restored when the document is opened again. Thanks to this feature, users can handle UTF-8 sources seamlessly as if pTeX interpreted UTF-8 source directly.

QUESTION: There is also a section "During File Save (for users in Japan)" with checkbox "Automatic UTF-8-Mac to UTF-8 Conversion". What does it do too?

ANSWER: In Japanese, some characters have two (or more) different Unicode representations. For example, the letter "が" can be represented in the following two ways.

- 1) が (U+304C, HIRAGANA LETTER GA) [composed form]
- 2) か (U+304B, HIRAGANA LETTER KA) + ゙(U+3099, COMBINING KATAKANA-HIRAGANA VOICED SOUND MARK) [decomposed form]

The composed form is more common and more widely used than the decomposed form. In fact, most Unix tools handle the character in the composed form. The Mac filesystem HFS+, however, stores filenames in the decomposed form.

Mixture of these representations causes a lot of troubles. For example, the following simple source causes the error "Undefined control sequence."

    \def\が{GA}
    \が

That's because the composed form is used in the first line and the decomposed form is used in the second line. However, they look exactly the same and it is impossible for normal users to distinguish them.

So I added the feature "Automatic UTF-8-Mac to UTF-8 Conversion" into TeXShop. If this is enabled, the decomposed form is converted into the composed form when the document is saved. This conversion is called Unicode normalization. This is useful to prevent troubles caused by the mixture of both representations.

The name "UTF-8-Mac" is a common name for the decomposed form. This name is derived from the fact that HFS+ adopts the decomposed form.

Strictly speaking, the normalization form that HFS+ adopts is a little different from standardized Unicode normalization forms like NFC or NFD. TeXShop adopts the same normalization form as HFS+, which is sometimes called modified-NFD.

I presented these features of TeXShop at the TUG 2013 annual conference. You can see the explanation of "utf-export" on pp. 26-28 in my slides https://tug.org/tug2013/slides/Terada-TeXShop.pdf>. Besides, you can get more detailed information about Unicode normalization on pp. 29-52.

ADDITIONAL INFORMATION: During the internet discussion about these modifications, additional historic information about pTeX was given by Yusuki Kuroki. Yusuke Terada then wrote "I am Yusuke Terada, a Japanese developer. Yusuke Kuroki is a friend of mine. He is one of the leading persons in Japanese TeX community. Please do not confuse these two Yusukes :-)"

Here is Yusuke Kuroki's elaboration:

Background:

- - pTeX was developed before Unicode-age; it operates with the JIS X 0208 character set (roughly speaking, a subset of Unicode) internally, and accepts files encoded by ISO-2022-JP, EUC-JP, or Shift_JIS (a related encoding of JIS X 0208 set).
- - Recent development extend the external character encoding for pTeX to UTF-8 (but internally it is JIS X 0208).
- - The otf package (that I often use in the sample files) was developed to input UTF/CID characters by Unicode point or CID such as \UTF{3251} or \CID{8091}.

  The support transliterates input UTF-8 characters outside JIS X 0208 to \UTF{...} representation. See Terada-san's slides https://tug.org/tug2013/slides/Terada-TeXShop.pdf>, (pp. 20--28).

Advanced Help
Japan