Skip to content

Commit

Permalink
deploy: be83b42
Browse files Browse the repository at this point in the history
  • Loading branch information
shelm committed Nov 23, 2024
0 parents commit 762a8d5
Show file tree
Hide file tree
Showing 427 changed files with 25,220 additions and 0 deletions.
4 changes: 4 additions & 0 deletions .buildinfo
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Sphinx build info version 1
# This file records the configuration used when building these files. When it is not found, a full rebuild will be done.
config: 42e691eb0377b4a9c3e0352ad88d5a0a
tags: 645f666f9bcd5a90fca523b33c5a78b7
4 changes: 4 additions & 0 deletions .buildinfo.bak
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Sphinx build info version 1
# This file records the configuration used when building these files. When it is not found, a full rebuild will be done.
config: 4f8ab8254e06004dc1691adb1cd5270a
tags: 645f666f9bcd5a90fca523b33c5a78b7
Binary file added .doctrees/api/data.doctree
Binary file not shown.
Binary file added .doctrees/api/data.handler.doctree
Binary file not shown.
Binary file added .doctrees/api/data.provider.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.app.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.backend.doctree
Binary file not shown.
Binary file not shown.
Binary file added .doctrees/api/discover.data.adjectives.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.data.animals.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.data.colors.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.data.countries.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.data.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.data.languages.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.data.names.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.data.star_wars.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.exec.doctree
Binary file not shown.
Binary file not shown.
Binary file added .doctrees/api/discover.route.cancel.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.route.cml_info.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.route.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.route.explain.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.route.extract.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.route.fetch_result.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.route.log.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.route.predict.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.route.process.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.route.status.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.route.train.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.route.ui.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.route.upload.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.templates.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.utils.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.utils.env.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.utils.import_utils.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.utils.job_utils.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.utils.log_utils.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.utils.thread_utils.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.utils.venv_utils.doctree
Binary file not shown.
Binary file not shown.
Binary file added .doctrees/api/discover_utils.data.data.doctree
Binary file not shown.
Binary file added .doctrees/api/discover_utils.data.doctree
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file added .doctrees/api/discover_utils.data.handler.doctree
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file added .doctrees/api/discover_utils.data.static.doctree
Binary file not shown.
Binary file added .doctrees/api/discover_utils.data.stream.doctree
Binary file not shown.
Binary file added .doctrees/api/discover_utils.doctree
Binary file not shown.
Binary file added .doctrees/api/discover_utils.interfaces.doctree
Binary file not shown.
Binary file not shown.
Binary file added .doctrees/api/discover_utils.scripts.doctree
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file added .doctrees/api/discover_utils.utils.doctree
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file added .doctrees/api/modules.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_utils.data.annotation.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_utils.data.data.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_utils.data.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_utils.data.handler.doctree
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file added .doctrees/api/nova_utils.data.provider.doctree
Binary file not shown.
Binary file not shown.
Binary file added .doctrees/api/nova_utils.data.session.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_utils.data.stream.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_utils.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_utils.interfaces.doctree
Binary file not shown.
Binary file not shown.
Binary file added .doctrees/api/nova_utils.scripts.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_utils.scripts.extract.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_utils.scripts.parsers.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_utils.scripts.predict.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_utils.scripts.process.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_utils.scripts.train.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_utils.storage.cache.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_utils.storage.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_utils.utils.anno_utils.doctree
Binary file not shown.
Binary file not shown.
Binary file added .doctrees/api/nova_utils.utils.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_utils.utils.json_utils.doctree
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file added .doctrees/environment.pickle
Binary file not shown.
Binary file added .doctrees/index.doctree
Binary file not shown.
Binary file added .doctrees/modules/blazeface.doctree
Binary file not shown.
Binary file added .doctrees/modules/diaristation.doctree
Binary file not shown.
Binary file added .doctrees/modules/facemesh.doctree
Binary file not shown.
Binary file added .doctrees/modules/libreface.doctree
Binary file not shown.
Binary file added .doctrees/modules/whisperx.doctree
Binary file not shown.
Binary file added .doctrees/tutorials/examples.doctree
Binary file not shown.
Binary file added .doctrees/tutorials/introduction.doctree
Binary file not shown.
Binary file added .doctrees/tutorials/lens.doctree
Binary file not shown.
Empty file added .nojekyll
Empty file.
235 changes: 235 additions & 0 deletions _modules/discover/utils/import_utils.html

Large diffs are not rendered by default.

288 changes: 288 additions & 0 deletions _modules/discover/utils/job_utils.html

Large diffs are not rendered by default.

218 changes: 218 additions & 0 deletions _modules/discover/utils/log_utils.html

Large diffs are not rendered by default.

213 changes: 213 additions & 0 deletions _modules/discover/utils/thread_utils.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,213 @@


<!DOCTYPE html>
<html class="writer-html5" lang="en" data-content_root="../../../">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>discover.utils.thread_utils &mdash; DISCOVER 1.0.1 documentation</title>
<link rel="stylesheet" type="text/css" href="../../../_static/pygments.css?v=80d5e7a1" />
<link rel="stylesheet" type="text/css" href="../../../_static/css/theme.css?v=e59714d7" />
<link rel="stylesheet" type="text/css" href="../../../_static/custom.css" />


<script src="../../../_static/jquery.js?v=5d32c60e"></script>
<script src="../../../_static/_sphinx_javascript_frameworks_compat.js?v=2cd50e6c"></script>
<script src="../../../_static/documentation_options.js?v=292eb321"></script>
<script src="../../../_static/doctools.js?v=9bcbadda"></script>
<script src="../../../_static/sphinx_highlight.js?v=dc90522c"></script>
<script src="../../../_static/js/theme.js"></script>
<link rel="index" title="Index" href="../../../genindex.html" />
<link rel="search" title="Search" href="../../../search.html" />
</head>

<body class="wy-body-for-nav">
<div class="wy-grid-for-nav">
<nav data-toggle="wy-nav-shift" class="wy-nav-side">
<div class="wy-side-scroll">
<div class="wy-side-nav-search" >



<a href="../../../index.html" class="icon icon-home">
DISCOVER
</a>
<div role="search">
<form id="rtd-search-form" class="wy-form" action="../../../search.html" method="get">
<input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
<p class="caption" role="heading"><span class="caption-text">Getting started</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../../tutorials/introduction.html">DISCOVER - A Modular Software Framework for Human Behavior Analysis</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../tutorials/examples.html">Examples</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../tutorials/lens.html">LENS: Learning and Exploring through Natural language Systems</a></li>
</ul>
<p class="caption" role="heading"><span class="caption-text">Modules</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../../modules/blazeface.html">BlazeFace</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../modules/facemesh.html">FaceMesh</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../modules/libreface.html">LibreFace</a></li>
</ul>
<p class="caption" role="heading"><span class="caption-text">Packages</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../../api/discover.app.html">discover.app module</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../api/discover.backend.html">discover.backend package</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../api/discover.exec.html">discover.exec package</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../api/discover.route.html">discover.route package</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../api/discover.utils.html">discover.utils package</a></li>
</ul>

</div>
</div>
</nav>

<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="Mobile navigation menu" >
<i data-toggle="wy-nav-top" class="fa fa-bars"></i>
<a href="../../../index.html">DISCOVER</a>
</nav>

<div class="wy-nav-content">
<div class="rst-content">
<div role="navigation" aria-label="Page navigation">
<ul class="wy-breadcrumbs">
<li><a href="../../../index.html" class="icon icon-home" aria-label="Home"></a></li>
<li class="breadcrumb-item"><a href="../../index.html">Module code</a></li>
<li class="breadcrumb-item active">discover.utils.thread_utils</li>
<li class="wy-breadcrumbs-aside">
</li>
</ul>
<hr/>
</div>
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
<div itemprop="articleBody">

<h1>Source code for discover.utils.thread_utils</h1><div class="highlight"><pre>
<span></span><span class="sd">&quot;&quot;&quot; Utility modules for NOVA-Server Threads</span>

<span class="sd">Author:</span>
<span class="sd"> Dominik Schiller &lt;[email protected]&gt;</span>
<span class="sd">Date:</span>
<span class="sd"> 13.09.2023</span>
<span class="sd">&quot;&quot;&quot;</span>

<span class="kn">import</span> <span class="nn">ctypes</span>
<span class="kn">import</span> <span class="nn">threading</span>

<span class="n">status_lock</span> <span class="o">=</span> <span class="n">threading</span><span class="o">.</span><span class="n">Lock</span><span class="p">()</span>
<span class="n">ml_lock</span> <span class="o">=</span> <span class="n">threading</span><span class="o">.</span><span class="n">Lock</span><span class="p">()</span>
<span class="n">jc_lock</span> <span class="o">=</span> <span class="n">threading</span><span class="o">.</span><span class="n">Lock</span><span class="p">()</span>
<span class="n">job_counter</span> <span class="o">=</span> <span class="mi">0</span>

<span class="n">THREADS</span> <span class="o">=</span> <span class="p">{}</span>


<div class="viewcode-block" id="ml_thread_wrapper">
<a class="viewcode-back" href="../../../api/discover.utils.thread_utils.html#discover.utils.thread_utils.ml_thread_wrapper">[docs]</a>
<span class="k">def</span> <span class="nf">ml_thread_wrapper</span><span class="p">(</span><span class="n">func</span><span class="p">):</span>
<span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
<span class="sd"> Executing the function in a mutex protected thread for asynchronous execution of long-running ml tasks.</span>
<span class="sd"> The thread waits with execution unit the status_lock is released. To do any initialization before the thread starts acquire the status lock before creating the thread.</span>
<span class="sd"> :param func:</span>
<span class="sd"> :return: The thread</span>
<span class="sd"> &quot;&quot;&quot;</span>

<span class="k">def</span> <span class="nf">wrapper</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="k">global</span> <span class="n">job_counter</span>

<span class="k">def</span> <span class="nf">lock</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="k">try</span><span class="p">:</span>
<span class="c1"># Only one ml thread can be active at the same time</span>
<span class="n">ml_lock</span><span class="o">.</span><span class="n">acquire</span><span class="p">()</span>
<span class="n">func</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
<span class="k">finally</span><span class="p">:</span>
<span class="n">ml_lock</span><span class="o">.</span><span class="n">release</span><span class="p">()</span>

<span class="n">jc_lock</span><span class="o">.</span><span class="n">acquire</span><span class="p">()</span>
<span class="n">job_id</span> <span class="o">=</span> <span class="nb">str</span><span class="p">(</span><span class="n">job_counter</span><span class="p">)</span>
<span class="n">job_counter</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="n">jc_lock</span><span class="o">.</span><span class="n">release</span><span class="p">()</span>

<span class="n">t</span> <span class="o">=</span> <span class="n">BackendThread</span><span class="p">(</span><span class="n">target</span><span class="o">=</span><span class="n">lock</span><span class="p">,</span> <span class="n">name</span><span class="o">=</span><span class="n">job_id</span><span class="p">,</span> <span class="n">args</span><span class="o">=</span><span class="n">args</span><span class="p">,</span> <span class="n">kwargs</span><span class="o">=</span><span class="n">kwargs</span><span class="p">)</span>
<span class="k">return</span> <span class="n">t</span>

<span class="k">return</span> <span class="n">wrapper</span></div>



<div class="viewcode-block" id="status_thread_wrapper">
<a class="viewcode-back" href="../../../api/discover.utils.thread_utils.html#discover.utils.thread_utils.status_thread_wrapper">[docs]</a>
<span class="k">def</span> <span class="nf">status_thread_wrapper</span><span class="p">(</span><span class="n">func</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">wrapper</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">status_lock</span><span class="o">.</span><span class="n">acquire</span><span class="p">()</span>
<span class="k">return</span> <span class="n">func</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
<span class="k">finally</span><span class="p">:</span>
<span class="n">status_lock</span><span class="o">.</span><span class="n">release</span><span class="p">()</span>

<span class="k">return</span> <span class="n">wrapper</span></div>



<div class="viewcode-block" id="BackendThread">
<a class="viewcode-back" href="../../../api/discover.utils.thread_utils.html#discover.utils.thread_utils.BackendThread">[docs]</a>
<span class="k">class</span> <span class="nc">BackendThread</span><span class="p">(</span><span class="n">threading</span><span class="o">.</span><span class="n">Thread</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">target</span><span class="p">,</span> <span class="n">name</span><span class="p">,</span> <span class="n">args</span><span class="p">,</span> <span class="n">kwargs</span><span class="p">):</span>
<span class="n">threading</span><span class="o">.</span><span class="n">Thread</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span>
<span class="bp">self</span><span class="p">,</span> <span class="n">target</span><span class="o">=</span><span class="n">target</span><span class="p">,</span> <span class="n">name</span><span class="o">=</span><span class="n">name</span><span class="p">,</span> <span class="n">args</span><span class="o">=</span><span class="n">args</span><span class="p">,</span> <span class="n">kwargs</span><span class="o">=</span><span class="n">kwargs</span>
<span class="p">)</span>

<div class="viewcode-block" id="BackendThread.get_id">
<a class="viewcode-back" href="../../../api/discover.utils.thread_utils.html#discover.utils.thread_utils.BackendThread.get_id">[docs]</a>
<span class="k">def</span> <span class="nf">get_id</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">if</span> <span class="nb">hasattr</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="s2">&quot;_thread_id&quot;</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_thread_id</span>
<span class="k">for</span> <span class="nb">id</span><span class="p">,</span> <span class="n">thread</span> <span class="ow">in</span> <span class="n">threading</span><span class="o">.</span><span class="n">_active</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
<span class="k">if</span> <span class="n">thread</span> <span class="ow">is</span> <span class="bp">self</span><span class="p">:</span>
<span class="k">return</span> <span class="nb">id</span></div>


<div class="viewcode-block" id="BackendThread.raise_exception">
<a class="viewcode-back" href="../../../api/discover.utils.thread_utils.html#discover.utils.thread_utils.BackendThread.raise_exception">[docs]</a>
<span class="k">def</span> <span class="nf">raise_exception</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">thread_id</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">get_id</span><span class="p">()</span>
<span class="n">res</span> <span class="o">=</span> <span class="n">ctypes</span><span class="o">.</span><span class="n">pythonapi</span><span class="o">.</span><span class="n">PyThreadState_SetAsyncExc</span><span class="p">(</span>
<span class="n">thread_id</span><span class="p">,</span> <span class="n">ctypes</span><span class="o">.</span><span class="n">py_object</span><span class="p">(</span><span class="ne">SystemExit</span><span class="p">)</span>
<span class="p">)</span>
<span class="k">if</span> <span class="n">res</span> <span class="o">&gt;</span> <span class="mi">1</span><span class="p">:</span>
<span class="n">ctypes</span><span class="o">.</span><span class="n">pythonapi</span><span class="o">.</span><span class="n">PyThreadState_SetAsyncExc</span><span class="p">(</span><span class="n">thread_id</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span></div>
</div>

</pre></div>

</div>
</div>
<footer>

<hr/>

<div role="contentinfo">
<p>&#169; Copyright 2023, Dominik Schiller.</p>
</div>

Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.


</footer>
</div>
</div>
</section>
</div>
<script>
jQuery(function () {
SphinxRtdTheme.Navigation.enable(true);
});
</script>

</body>
</html>
Loading

0 comments on commit 762a8d5

Please sign in to comment.