Skip to content

Commit

Permalink
deploy: c521cba
Browse files Browse the repository at this point in the history
  • Loading branch information
shelm committed Oct 10, 2024
0 parents commit 046055b
Show file tree
Hide file tree
Showing 534 changed files with 28,481 additions and 0 deletions.
4 changes: 4 additions & 0 deletions .buildinfo
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Sphinx build info version 1
# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
config: e9b96dda5391161b0544b06434a56076
tags: 645f666f9bcd5a90fca523b33c5a78b7
Binary file added .doctrees/api/data.doctree
Binary file not shown.
Binary file added .doctrees/api/data.handler.doctree
Binary file not shown.
Binary file added .doctrees/api/data.provider.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.app.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.backend.doctree
Binary file not shown.
Binary file not shown.
Binary file added .doctrees/api/discover.data.adjectives.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.data.animals.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.data.colors.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.data.countries.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.data.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.data.languages.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.data.names.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.data.star_wars.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.exec.doctree
Binary file not shown.
Binary file not shown.
Binary file added .doctrees/api/discover.route.cancel.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.route.cml_info.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.route.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.route.explain.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.route.extract.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.route.fetch_result.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.route.log.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.route.predict.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.route.process.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.route.status.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.route.train.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.route.ui.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.route.upload.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.templates.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.utils.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.utils.env.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.utils.import_utils.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.utils.job_utils.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.utils.log_utils.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.utils.thread_utils.doctree
Binary file not shown.
Binary file added .doctrees/api/discover.utils.venv_utils.doctree
Binary file not shown.
Binary file not shown.
Binary file added .doctrees/api/discover_utils.data.data.doctree
Binary file not shown.
Binary file added .doctrees/api/discover_utils.data.doctree
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file added .doctrees/api/discover_utils.data.handler.doctree
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file added .doctrees/api/discover_utils.data.static.doctree
Binary file not shown.
Binary file added .doctrees/api/discover_utils.data.stream.doctree
Binary file not shown.
Binary file added .doctrees/api/discover_utils.doctree
Binary file not shown.
Binary file added .doctrees/api/discover_utils.interfaces.doctree
Binary file not shown.
Binary file not shown.
Binary file added .doctrees/api/discover_utils.scripts.doctree
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file added .doctrees/api/discover_utils.utils.doctree
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file added .doctrees/api/modules.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_server.app.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_server.backend.doctree
Binary file not shown.
Binary file not shown.
Binary file added .doctrees/api/nova_server.data.adjectives.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_server.data.animals.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_server.data.colors.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_server.data.countries.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_server.data.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_server.data.languages.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_server.data.names.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_server.data.star_wars.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_server.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_server.exec.doctree
Binary file not shown.
Binary file not shown.
Binary file added .doctrees/api/nova_server.route.cancel.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_server.route.cml_info.doctree
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file added .doctrees/api/nova_server.route.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_server.route.explain.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_server.route.extract.doctree
Binary file not shown.
Binary file not shown.
Binary file added .doctrees/api/nova_server.route.log.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_server.route.predict.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_server.route.process.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_server.route.status.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_server.route.train.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_server.route.ui.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_server.templates.doctree
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file added .doctrees/api/nova_server.utils.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_server.utils.env.doctree
Binary file not shown.
Binary file not shown.
Binary file added .doctrees/api/nova_server.utils.job_utils.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_server.utils.log_utils.doctree
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file added .doctrees/api/nova_utils.data.annotation.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_utils.data.data.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_utils.data.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_utils.data.handler.doctree
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file added .doctrees/api/nova_utils.data.provider.doctree
Binary file not shown.
Binary file not shown.
Binary file added .doctrees/api/nova_utils.data.session.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_utils.data.stream.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_utils.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_utils.interfaces.doctree
Binary file not shown.
Binary file not shown.
Binary file added .doctrees/api/nova_utils.scripts.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_utils.scripts.extract.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_utils.scripts.parsers.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_utils.scripts.predict.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_utils.scripts.process.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_utils.scripts.train.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_utils.storage.cache.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_utils.storage.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_utils.utils.anno_utils.doctree
Binary file not shown.
Binary file not shown.
Binary file added .doctrees/api/nova_utils.utils.doctree
Binary file not shown.
Binary file added .doctrees/api/nova_utils.utils.json_utils.doctree
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file added .doctrees/environment.pickle
Binary file not shown.
Binary file added .doctrees/index.doctree
Binary file not shown.
Binary file added .doctrees/modules/blazeface_link.doctree
Binary file not shown.
Binary file added .doctrees/modules/overview_link.doctree
Binary file not shown.
Binary file added .doctrees/modules/whisperx_link.doctree
Binary file not shown.
Binary file added .doctrees/tutorials/examples.doctree
Binary file not shown.
Binary file added .doctrees/tutorials/introduction.doctree
Binary file not shown.
Empty file added .nojekyll
Empty file.
227 changes: 227 additions & 0 deletions _modules/discover/utils/import_utils.html

Large diffs are not rendered by default.

280 changes: 280 additions & 0 deletions _modules/discover/utils/job_utils.html

Large diffs are not rendered by default.

210 changes: 210 additions & 0 deletions _modules/discover/utils/log_utils.html

Large diffs are not rendered by default.

205 changes: 205 additions & 0 deletions _modules/discover/utils/thread_utils.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,205 @@


<!DOCTYPE html>
<html class="writer-html5" lang="en" data-content_root="../../../">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>discover.utils.thread_utils &mdash; DISCOVER 1.0.0 documentation</title>
<link rel="stylesheet" type="text/css" href="../../../_static/pygments.css?v=80d5e7a1" />
<link rel="stylesheet" type="text/css" href="../../../_static/css/theme.css?v=e59714d7" />


<script src="../../../_static/jquery.js?v=5d32c60e"></script>
<script src="../../../_static/_sphinx_javascript_frameworks_compat.js?v=2cd50e6c"></script>
<script src="../../../_static/documentation_options.js?v=8d563738"></script>
<script src="../../../_static/doctools.js?v=9a2dae69"></script>
<script src="../../../_static/sphinx_highlight.js?v=dc90522c"></script>
<script src="../../../_static/js/theme.js"></script>
<link rel="index" title="Index" href="../../../genindex.html" />
<link rel="search" title="Search" href="../../../search.html" />
</head>

<body class="wy-body-for-nav">
<div class="wy-grid-for-nav">
<nav data-toggle="wy-nav-shift" class="wy-nav-side">
<div class="wy-side-scroll">
<div class="wy-side-nav-search" >



<a href="../../../index.html" class="icon icon-home">
DISCOVER
</a>
<div role="search">
<form id="rtd-search-form" class="wy-form" action="../../../search.html" method="get">
<input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
<p class="caption" role="heading"><span class="caption-text">Getting started</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../../tutorials/introduction.html">DISCOVER - A Modular Software Framework for Human Behavior Analysis</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../tutorials/examples.html">Examples</a></li>
</ul>
<p class="caption" role="heading"><span class="caption-text">Packages</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../../api/discover.app.html">discover.app module</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../api/discover.backend.html">discover.backend package</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../api/discover.exec.html">discover.exec package</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../api/discover.route.html">discover.route package</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../api/discover.utils.html">discover.utils package</a></li>
</ul>

</div>
</div>
</nav>

<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="Mobile navigation menu" >
<i data-toggle="wy-nav-top" class="fa fa-bars"></i>
<a href="../../../index.html">DISCOVER</a>
</nav>

<div class="wy-nav-content">
<div class="rst-content">
<div role="navigation" aria-label="Page navigation">
<ul class="wy-breadcrumbs">
<li><a href="../../../index.html" class="icon icon-home" aria-label="Home"></a></li>
<li class="breadcrumb-item"><a href="../../index.html">Module code</a></li>
<li class="breadcrumb-item active">discover.utils.thread_utils</li>
<li class="wy-breadcrumbs-aside">
</li>
</ul>
<hr/>
</div>
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
<div itemprop="articleBody">

<h1>Source code for discover.utils.thread_utils</h1><div class="highlight"><pre>
<span></span><span class="sd">&quot;&quot;&quot; Utility modules for NOVA-Server Threads</span>

<span class="sd">Author:</span>
<span class="sd"> Dominik Schiller &lt;[email protected]&gt;</span>
<span class="sd">Date:</span>
<span class="sd"> 13.09.2023</span>
<span class="sd">&quot;&quot;&quot;</span>

<span class="kn">import</span> <span class="nn">ctypes</span>
<span class="kn">import</span> <span class="nn">threading</span>

<span class="n">status_lock</span> <span class="o">=</span> <span class="n">threading</span><span class="o">.</span><span class="n">Lock</span><span class="p">()</span>
<span class="n">ml_lock</span> <span class="o">=</span> <span class="n">threading</span><span class="o">.</span><span class="n">Lock</span><span class="p">()</span>
<span class="n">jc_lock</span> <span class="o">=</span> <span class="n">threading</span><span class="o">.</span><span class="n">Lock</span><span class="p">()</span>
<span class="n">job_counter</span> <span class="o">=</span> <span class="mi">0</span>

<span class="n">THREADS</span> <span class="o">=</span> <span class="p">{}</span>


<div class="viewcode-block" id="ml_thread_wrapper">
<a class="viewcode-back" href="../../../api/discover.utils.thread_utils.html#discover.utils.thread_utils.ml_thread_wrapper">[docs]</a>
<span class="k">def</span> <span class="nf">ml_thread_wrapper</span><span class="p">(</span><span class="n">func</span><span class="p">):</span>
<span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
<span class="sd"> Executing the function in a mutex protected thread for asynchronous execution of long-running ml tasks.</span>
<span class="sd"> The thread waits with execution unit the status_lock is released. To do any initialization before the thread starts acquire the status lock before creating the thread.</span>
<span class="sd"> :param func:</span>
<span class="sd"> :return: The thread</span>
<span class="sd"> &quot;&quot;&quot;</span>

<span class="k">def</span> <span class="nf">wrapper</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="k">global</span> <span class="n">job_counter</span>

<span class="k">def</span> <span class="nf">lock</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="k">try</span><span class="p">:</span>
<span class="c1"># Only one ml thread can be active at the same time</span>
<span class="n">ml_lock</span><span class="o">.</span><span class="n">acquire</span><span class="p">()</span>
<span class="n">func</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
<span class="k">finally</span><span class="p">:</span>
<span class="n">ml_lock</span><span class="o">.</span><span class="n">release</span><span class="p">()</span>

<span class="n">jc_lock</span><span class="o">.</span><span class="n">acquire</span><span class="p">()</span>
<span class="n">job_id</span> <span class="o">=</span> <span class="nb">str</span><span class="p">(</span><span class="n">job_counter</span><span class="p">)</span>
<span class="n">job_counter</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="n">jc_lock</span><span class="o">.</span><span class="n">release</span><span class="p">()</span>

<span class="n">t</span> <span class="o">=</span> <span class="n">BackendThread</span><span class="p">(</span><span class="n">target</span><span class="o">=</span><span class="n">lock</span><span class="p">,</span> <span class="n">name</span><span class="o">=</span><span class="n">job_id</span><span class="p">,</span> <span class="n">args</span><span class="o">=</span><span class="n">args</span><span class="p">,</span> <span class="n">kwargs</span><span class="o">=</span><span class="n">kwargs</span><span class="p">)</span>
<span class="k">return</span> <span class="n">t</span>

<span class="k">return</span> <span class="n">wrapper</span></div>



<div class="viewcode-block" id="status_thread_wrapper">
<a class="viewcode-back" href="../../../api/discover.utils.thread_utils.html#discover.utils.thread_utils.status_thread_wrapper">[docs]</a>
<span class="k">def</span> <span class="nf">status_thread_wrapper</span><span class="p">(</span><span class="n">func</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">wrapper</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">status_lock</span><span class="o">.</span><span class="n">acquire</span><span class="p">()</span>
<span class="k">return</span> <span class="n">func</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
<span class="k">finally</span><span class="p">:</span>
<span class="n">status_lock</span><span class="o">.</span><span class="n">release</span><span class="p">()</span>

<span class="k">return</span> <span class="n">wrapper</span></div>



<div class="viewcode-block" id="BackendThread">
<a class="viewcode-back" href="../../../api/discover.utils.thread_utils.html#discover.utils.thread_utils.BackendThread">[docs]</a>
<span class="k">class</span> <span class="nc">BackendThread</span><span class="p">(</span><span class="n">threading</span><span class="o">.</span><span class="n">Thread</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">target</span><span class="p">,</span> <span class="n">name</span><span class="p">,</span> <span class="n">args</span><span class="p">,</span> <span class="n">kwargs</span><span class="p">):</span>
<span class="n">threading</span><span class="o">.</span><span class="n">Thread</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span>
<span class="bp">self</span><span class="p">,</span> <span class="n">target</span><span class="o">=</span><span class="n">target</span><span class="p">,</span> <span class="n">name</span><span class="o">=</span><span class="n">name</span><span class="p">,</span> <span class="n">args</span><span class="o">=</span><span class="n">args</span><span class="p">,</span> <span class="n">kwargs</span><span class="o">=</span><span class="n">kwargs</span>
<span class="p">)</span>

<div class="viewcode-block" id="BackendThread.get_id">
<a class="viewcode-back" href="../../../api/discover.utils.thread_utils.html#discover.utils.thread_utils.BackendThread.get_id">[docs]</a>
<span class="k">def</span> <span class="nf">get_id</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">if</span> <span class="nb">hasattr</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="s2">&quot;_thread_id&quot;</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_thread_id</span>
<span class="k">for</span> <span class="nb">id</span><span class="p">,</span> <span class="n">thread</span> <span class="ow">in</span> <span class="n">threading</span><span class="o">.</span><span class="n">_active</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
<span class="k">if</span> <span class="n">thread</span> <span class="ow">is</span> <span class="bp">self</span><span class="p">:</span>
<span class="k">return</span> <span class="nb">id</span></div>


<div class="viewcode-block" id="BackendThread.raise_exception">
<a class="viewcode-back" href="../../../api/discover.utils.thread_utils.html#discover.utils.thread_utils.BackendThread.raise_exception">[docs]</a>
<span class="k">def</span> <span class="nf">raise_exception</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">thread_id</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">get_id</span><span class="p">()</span>
<span class="n">res</span> <span class="o">=</span> <span class="n">ctypes</span><span class="o">.</span><span class="n">pythonapi</span><span class="o">.</span><span class="n">PyThreadState_SetAsyncExc</span><span class="p">(</span>
<span class="n">thread_id</span><span class="p">,</span> <span class="n">ctypes</span><span class="o">.</span><span class="n">py_object</span><span class="p">(</span><span class="ne">SystemExit</span><span class="p">)</span>
<span class="p">)</span>
<span class="k">if</span> <span class="n">res</span> <span class="o">&gt;</span> <span class="mi">1</span><span class="p">:</span>
<span class="n">ctypes</span><span class="o">.</span><span class="n">pythonapi</span><span class="o">.</span><span class="n">PyThreadState_SetAsyncExc</span><span class="p">(</span><span class="n">thread_id</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span></div>
</div>

</pre></div>

</div>
</div>
<footer>

<hr/>

<div role="contentinfo">
<p>&#169; Copyright 2023, Dominik Schiller.</p>
</div>

Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.


</footer>
</div>
</div>
</section>
</div>
<script>
jQuery(function () {
SphinxRtdTheme.Navigation.enable(true);
});
</script>

</body>
</html>
Loading

0 comments on commit 046055b

Please sign in to comment.