DocsTranslationFileFinder

#doctest: +ELLIPSIS .. _translation_file_finder:

Translation File Finder

This tool provides a way of mapping translation files in a filesystem to Pootle codes for Language and Directory as well as filename and extension.

For a given translation path it will scan a filesystem looking for matching files, and return the matching information.

Translation paths are given in the format

/path/to/my/translation/resources/<directory_path>/<lang>/<filename>.<ext>

At a minimum a path must have a <lang> tag and end with .<ext>

Paths are matched using a regex, and the directory_path, lang and filename matches can be read from the results.

Retrieving a Translation file finder

The finder should be given an absolute file path.

>>> from pootle_fs.finder import TranslationFileFinder
>>> finder = TranslationFileFinder("/path/to/<lang>.<ext>")
>>> finder
<pootle_fs.finder.TranslationFileFinder object at ...>

Matching a simple filepath

This will match a file and interpret the filename without the ext as the language, if the file matches.

If the <filename> tag is not used filename and language are both set with the parsed path.

>>> finder.match_filepath("/path/to/foo.po")
('/path/to/foo.po', {'lang': 'foo', 'ext': 'po', 'filename': 'foo', 'directory_path': ''})

>>> finder.match_filepath("/path/to/some-foo.po")
('/path/to/some-foo.po', {'lang': 'some-foo', 'ext': 'po', 'filename': 'some-foo', 'directory_path': ''})

>>> finder.match_filepath("/path/to/another_foo.po")
('/path/to/another_foo.po', {'lang': 'another_foo', 'ext': 'po', 'filename': 'another_foo', 'directory_path': ''})

The translation path does not include any subdirectories, so this does not match:

>>> finder.match_filepath("/path/to/bar/foo.po") is None
True

Matching a filepath with filename

With the following examples, the matcher will look for a section to use in Pootle as the filename

>>> finder = TranslationFileFinder("/path/to/<lang>/<filename>.<ext>")

This will not match, because it must match both the lang and filename.

>>> finder.match_filepath("/path/to/foo.po") is None
True

But this will match both filename and lang.

>>> finder.match_filepath("/path/to/foo/bar.po")
('/path/to/foo/bar.po', {'lang': 'foo', 'ext': 'po', 'directory_path': '', 'filename': 'bar'})

Translation paths can be structured according to requirements

>>> finder = TranslationFileFinder("/path/to/<lang>/some/other/<filename>.<ext>")

>>> finder.match_filepath("/path/to/foo/bar.po") is None
True

>>> finder.match_filepath("/path/to/foo/some/other/bar.po")
('/path/to/foo/some/other/bar.po', {'lang': 'foo', 'ext': 'po', 'directory_path': '', 'filename': 'bar'})

The elements can appear in any order, and anywhere in the translation path.

>>> finder = TranslationFileFinder("/path/to/<filename>/some/<lang>/other.<ext>")
>>> finder.match_filepath("/path/to/foo/some/bar/other.po")
('/path/to/foo/some/bar/other.po', {'lang': 'bar', 'ext': 'po', 'directory_path': '', 'filename': 'foo'})

>>> finder = TranslationFileFinder("/path/to/some-<filename>/other-<lang>/filename.<ext>")
>>> finder.match_filepath("/path/to/some-foo/other-bar/filename.po")
('/path/to/some-foo/other-bar/filename.po', {'lang': 'bar', 'ext': 'po', 'directory_path': '', 'filename': 'foo'})

Matching a directory path

You can also match a directory path.

>>> finder = TranslationFileFinder("/path/to/<directory_path>/<lang>.<ext>")
>>> finder.match_filepath("/path/to/some/foo.po")
('/path/to/some/foo.po', {'lang': 'foo', 'ext': 'po', 'filename': 'foo', 'directory_path': 'some'})

This can include ``/``s.

>>> finder.match_filepath("/path/to/some/other/foo.po")
('/path/to/some/other/foo.po', {'lang': 'foo', 'ext': 'po', 'filename': 'foo', 'directory_path': 'some/other'})

And you can match a filename at the same time

>>> finder = TranslationFileFinder("/path/to/<directory_path>/<lang>/translation-file.<ext>")
>>> finder.match_filepath("/path/to/some/foo/translation-file.po")
('/path/to/some/foo/translation-file.po', {'lang': 'foo', 'ext': 'po', 'filename': 'translation-file', 'directory_path': 'some'})

>>> finder.match_filepath("/path/to/some/other/foo/translation-file.po")
('/path/to/some/other/foo/translation-file.po', {'lang': 'foo', 'ext': 'po', 'filename': 'translation-file', 'directory_path': 'some/other'})

Filtering with glob filters

You can use glob style pattern matching to filter matches.

>>> finder = TranslationFileFinder(
...     "/path/to/<directory_path>/<lang>/translation-file.<ext>",
...     filter_path="/path/to/dir1/*")


>>> finder.match_filepath("/path/to/dir1/foo/translation-file.po")
('/path/to/dir1/foo/translation-file.po', {'lang': 'foo', 'ext': 'po', 'filename': 'translation-file', 'directory_path': 'dir1'})

Which would exclude dir2

>>> finder.match_filepath("/path/to/dir2/foo/translation-file.po") is None
True

File extensions

By default the matcher will match either .po or .pot

>>> finder = TranslationFileFinder("/path/to/<lang>.<ext>")

>>> finder.match_filepath("/path/to/foo.po")
('/path/to/foo.po', {'lang': 'foo', 'ext': 'po', 'filename': 'foo', 'directory_path': ''})

>>> finder.match_filepath("/path/to/foo.pot")
('/path/to/foo.pot', {'lang': 'foo', 'ext': 'pot', 'filename': 'foo', 'directory_path': ''})

But you can match alternate extensions if you require.

>>> finder = TranslationFileFinder("/path/to/<lang>.<ext>", ext=["abc", "xyz"])

>>> finder.match_filepath("/path/to/foo.po") is None
True

>>> finder.match_filepath("/path/to/foo.abc")
('/path/to/foo.abc', {'lang': 'foo', 'ext': 'abc', 'filename': 'foo', 'directory_path': ''})

>>> finder.match_filepath("/path/to/foo.xyz")
('/path/to/foo.xyz', {'lang': 'foo', 'ext': 'xyz', 'filename': 'foo', 'directory_path': ''})

Walking a file-system

Let's create a temporary file

>>> import os
>>> import tempfile
>>> tmpdir = tempfile.mkdtemp()
>>> open(os.path.join(tmpdir, "foo.po"), "w").write("")
>>> open(os.path.join(tmpdir, "bar.po"), "w").write("")
>>> open(os.path.join(tmpdir, "baz.xliff"), "w").write("")
>>> finder = TranslationFileFinder(os.path.join(str(tmpdir), "<lang>.<ext>"))

The finder with ascertain the file root, i.e. the directory that it should look in.

>>> finder.file_root == str(tmpdir)
True

You can walk the file-system from the finder.file_root. This will return the file-paths regardless of whether they match.

>>> finder.walk()
<generator object walk at ...>

>>> list(finder.walk()) == [os.path.join(tmpdir, f) for f in ["bar.po", "baz.xliff", "foo.po"]]
True

To find matching files, use finder.find

>>> finder.find()
<generator object find at ...>

It should just find the 2 po files

>>> len(list(finder.find()))
2

>>> os.path.basename(list(finder.find())[0][0])
'bar.po'

Reverse matching

Given a language, and optionally directory_path, filename, and ext the finder can provide a reverse path according to its translation_path

It will use the first ext from the extension list, the default is .po

>>> finder = TranslationFileFinder("/path/to/<lang>.<ext>")

>>> finder.reverse_match("foo")
'/path/to/foo.po'

You can map it to another extension in the finders extensions list, by default this includes .pot

>>> finder.reverse_match("foo", ext="pot")
'/path/to/foo.pot'

But you cant use an ext not in the finders list

>>> import pytest

>>> with pytest.raises(ValueError):
...     finder.reverse_match("foo", ext="abc")

If the translation_path includes filename but you don`t specify it the matcher will reuse the lang

>>> finder = TranslationFileFinder("/path/to/<lang>/<filename>.<ext>")
>>> finder.reverse_match("foo")
'/path/to/foo/foo.po'

Or you can set it as required

>>> finder.reverse_match("foo", filename="bar", ext="pot")
'/path/to/foo/bar.pot'

And you can set the directory_path too

>>> finder = TranslationFileFinder("/path/to/<directory_path>/<lang>/<filename>.<ext>")

Which is optional when reverse matching

>>> finder.reverse_match("foo", filename="bar", ext="pot")
'/path/to/foo/bar.pot'

>>> finder.reverse_match("foo", filename="bar", ext="pot", directory_path="some/other")
'/path/to/some/other/foo/bar.pot'