Btukfyl

There is a glaring design flaw in TeX or Syntex for location of words.

Given a text in a .tex files:

Blah blah blah. This is a sentence blah! Blah Blah!

The whole paragraph is treated as one unit rather than the individual words.

It turns out that if one adds new lines to each word or line then the syncing works great:

Blah 

blah 

blah. 

This 

is 

a 

sentence 

blah! 

Blah 

Blah!

And we get single word accuracy.

The problem is that it makes reading the tex file pointless and hence it is not a proper solution.

A real solution would be to have the compiler to automatically translate words(or lines) in to separate lines automatically.

This would provide the proper syntex for word or line resolution but the back propagation would not match the original unaltered .tex file.

The line info would need to be kept track of and then inversely mapped.

Is there any way to do this easily?

e.g.,

Line1:

Blah blah blah. This is a sentence blah! Blah Blah!

Line2:

becomes
Line1:
Blah
Line 2:
blah
Line 3:
blah.
Line 4:
This
Line 5:
is
Line 6:
a
Line 7:
sentence
Line 8:
blah!
Line 9:
Blah
Line 10:
Blah!

And these inserted new lines positions are then kept track of to do the inverse map back in to the column that they were inserted from.

This is all basic mapping and pretty easy do to with a few lines of code except for a few problems.

1: One must not insert new lines in invalid place that will break the .tex file or alter it's output.
2: The editor that responds to the synctex must be able to do the inverse mapping from lines to line/col.

Essentially one could insert a new line in every space as long as it won't break the .tex code and would get word level accuracy. This could potentially be done pre-compilation if proper word boundaries could be determined. I think though it would be better if it could be done in an environment such as

begin{synctexing}

end{synctexing}

which would essentially add the new lines to all the text in between and ignore any environments inside so things like tikz pictures could be inserted without having to create a bunch of environments.

It would simply output the mapping file(line/col to line) and the editor would have to be able to reverse the map(easy but it would require supporting it unless they already support columns(but that synctex just always gives them 0).

The problem is not that this is difficult but about the specifics which I know nothing about.

In fact, even if we had to do something like snl which adds a new line to the synctex, it would be better than nothing.

The SyncTeX architecture currently maps the position on the PDF page to the line and a “tag” in the input file (I don't really know what “tag“ is).

This is the SyncTeX data structure from syntex.c:

/*  Here are all the local variables gathered in one "synchronization context"  */

static struct {

    void *file;                 /*  the foo.synctex or foo.synctex.gz I/O identifier  */

    synctex_fprintf_t fprintf;  /*  either fprintf or gzprintf */

    char *busy_name;            /*  the real "foo.synctex(busy)" or "foo.synctex.gz(busy)" name, with output_directory  */

    char *root_name;            /*  in general jobname.tex  */

    integer count;              /*  The number of interesting records in "foo.synctex"  */

    /*  next concern the last sync record encountered  */

    halfword node;              /*  the last synchronized node, must be set 

                                 *  before the recorder */

    synctex_recorder_t recorder;/*  the recorder of the node above, the

                                 *  routine that knows how to record the 

                                 *  node to the .synctex file */

    integer tag, line;          /*  current tag and line  */

    integer curh, curv;         /*  current point  */

    integer magnification;      /*  The magnification as given by mag */

    integer unit;               /*  The unit, defaults to 1, use 8192 to produce shorter but less accurate info */

    integer total_length;       /*  The total length of the bytes written since    the last check point  */

    integer options;            /* unsigned options */

    integer lastv;              /* compression trick if

                                 |synctex_options&4|>0.  */

    integer form_depth;        /*  pdf forms are an example of nested sheets */

    struct _flags {

        unsigned int option_read:1; /*  Command line option read (in case of problem or at the end) */

        unsigned int content_ready:1; /*  Command line option read (in case of problem or at the end) */

        unsigned int off:1;         /*  Definitely turn off synctex, corresponds to cli option -synctex=0 */

        unsigned int no_gz:1;       /*  Whether zlib is used or not */

        unsigned int not_void:1;    /*  Whether it really contains synchronization material */

        unsigned int warn:1;        /*  One shot warning flag */

        unsigned int quoted:1;      /*  Whether the input file name was quoted by tex or not, for example ""my input file.tex"", unused by XeTeX */

        unsigned int output_p:1;    /*  Whether the output_directory is used */

        unsigned int reserved:SYNCTEX_BITS_PER_BYTE*sizeof(int)-8; /* Align */

    } flags;

} synctex_ctxt = {

To also record the column in the input file, you'd have to add another field to the data structure and adapt all the recorder functions to write this entry to the SyncTeX file. The next step would be to adapt the SyncTeX parser to pick up the column from the SyncTeX file. Then you'd have to edit the source code of your editor to make it aware that inverse search now also maps to the column. The hardest part is, that all of the above has to be done in a backwards compatible way, such that your editor can still read SyncTeX files from older version and that SyncTeX files written with the newer version can still be read by old parsers.

So bottomlined, it is theoretically possible to implement your idea but finding someone to practically do this is probably NP-hard.