fortls gets confused when the code includes form feed (’\f’, ASCII 0xC)

[tomoto]

First of all, thank you for all your efforts put into this useful project! I'm posting a report about the problem I found (and its solution) hoping it may help others as well.

Describe the bug

When the code has form feed characters (’\f’, ASCII 0xC), most language support features including mouse hover, go to definition/references, etc. do not function after the form feed characters. In the example below, fortls failed to find the definition of "ABC".

The cause is rather obvious. When fortls parses the code, the form feed characters are recognized as one of the line separators. It results in the wrong line splits as shown below.

In the example above, when the client asks for the information about "ABC" in line 4, fortls cannot find anything because "ABC" actually (and incorrectly) exists in the next line in the internal data.

Just in case you ask why this matters, you often need to handle this situation when dealing with historical code. gfortran also handles it as below (https://gcc.gnu.org/onlinedocs/gfortran/Form-feed-as-whitespace.html ):

5.1.26 Form feed as whitespace
Historically, legacy compilers allowed insertion of form feed characters (’\f’, ASCII 0xC) at the beginning of lines for formatted output to line printers, though the Fortran standard does not mention this. GNU Fortran supports the interpretation of form feed characters in source as whitespace for compatibility.

To Reproduce

1. Open a code that includes one or more form feed characters.
2. Perform mouse hover or go to definition/references on some variable/function/subroutine name below at least one form feed character.
3. Observe you will get nothing or wrong information.

Expected behavior

The correct information should be shown (as the case where no form feed characters exist).

Internally, the line split should be exactly the same as VSCode.

Screenshots & Animations

Nothing to add.

Setup information (please complete the following information):

• OS: Linux
• Python Version: 3.10.12
• fortls Version: 3.1.3
• Code editor used: VS Code
• the Fortran extension for the code editor and its version: Modern Fortran v3.2.0

Configuration information (please complete the following information):

Nothing special.

Additional context

I will post my suggested fix as a PR after making sure everything is alright. Please give me some time.