Project configuration file not loaded V2.1

[eradin]

Hi, this was an issue with V1.7 but I am seeing something similar with 2.1.

Reference: #108

I have installed crunz and uploaded to my server. The test server is a shared hosted environment (linux) running php 7.2. I have set up cron per the instructions with a copy of the config file in my project root folder. When I run the command from cron with debugging level 3, I get an error

No task found! Please check your source path. Additional output show it can't find the config file in ..lavary/crunz.

When I run this same command from the cli, it works perfectly. Finding the config in the root along with the tasks in the tasks directory (also in the root).

I did a little testing and discovered the first script file bin/crunz shows the cwd as the project root but then changes the directory to call the second script in lavary/crunz. When I print the cwd from the second script it shows the cwd as ..lavary/crunz. When I run from the cli, both files cwd's are the project root.

Same scripts, same command. The cli method works as advertised but the cron method fails due to an erroneous current directory. Obviously the cron version is more important.

Is this a bug? I was able to patch it up by adding a symbolic link in the lavary/crunz directory to point to the config.yml in the root. Then I added the path to the task directory in the cron command line. This works but it's very kludgy (not to mention I had to make a lot of paths absolute in my task file).

Has anybody seen this before?

[PabloKowalczyk]

Hello,

in v2.1 Crunz will only look for crunz.yml in current working directory (https://github.com/lavary/crunz/blob/master/UPGRADE.md#upgrading-from-v110-to-v111).

Could you show me result of vendor/bin/crunz schedule:run -vvv and your Cron entry?

[eradin]

Thank you for the feedback. I am doing everything according to the docs. Here is some more information.

cron job:
cd /home/relevant/public_html && vendor/bin/crunz schedule:run -vvv > tasks.log 2>&1

directory structure
public_html/
-crunz.yml
-tasks/
-vendor/
--bin/
---crunz

Output of run:
Config file not found, exception message: 'Unable to find config file "/home/relevant/public_html/vendor/lavary/crunz/crunz.yml".'.
Unable to find/parse config file, fallback to default values.
Task source path '/home/relevant/public_html/vendor/lavary/crunz/tasks'
No task found! Please check your source path.
Content-type: text/html; charset=UTF-8

As you can see, it does change directory from the cron command but when it runs lavary/crunz, it thinks it's in that directory not the project root.

[davidsneighbour]

In my experience on shared hosting you should use the full path and call php, like the following:

cd /home/relevant/public_html && php ./vendor/bin/crunz schedule:run -vvv > tasks.log 2>&1

./vendor/bin/crunz might have to be even /home/relevant/public_html/vendor/bin/crunz

The following works for me in my crontab:

* * * * * cd /home/username/webroot && /usr/local/php73/bin/php vendor/bin/crunz schedule:run

[PabloKowalczyk]

@davidsneighbour good tip, using absolute path is always better option.

@eradin Here is output for command cd /var/www/html && vendor/bin/crunz s:l -vvv in my project (missing crunz.yml on purpose):

Unable to find/parse config file, fallback to default values.
Task source path '/var/www/html/tasks'
Task finder suffix: 'Tasks.php'
Realpath for '/var/www/html/tasks' is '/var/www/html/tasks'
Found 1 task(s) at path '/var/www/html/tasks'

Everything works as expected. Are you sure Crunz version (vendor/bin/crunz --version) on server is v2.1?

[eradin]

I just checked again. --version show V2.1.

I also tried absolute path:

cd /home/relevant/public_html && /home/relevant/public_html/vendor/bin/crunz schedule:run -vvv > tasks.log 2>&1

tasks.log:
Config file not found, exception message: 'Unable to find config file "/home/relevant/public_html/vendor/lavary/crunz/crunz.yml".'.
Unable to find/parse config file, fallback to default values.
Task source path '/home/relevant/public_html/vendor/lavary/crunz/tasks'
No task found! Please check your source path.
Content-type: text/html; charset=UTF-8

I will repeat, this is only a problem from cron. Running from shell works fine. It's also a shared hosting environment which could be significant.

Observation: In the bin/crunz file, the first line is
dir=$(cd "${0%[/\]*}" > /dev/null; cd "../lavary/crunz" && pwd)

Could that 2nd cd be causing the problem? If so, why would it work differently on cron vs the cli?

UPDATE
I replaced the lines in the bin/crunz file with an absolute path

/home/relevant/public_html/vendor/lavary/crunz/crunz "$@"

Oddly, the tasks.log file is unchanged. So who's changing the root directory?

[eradin]

A little more information. I added a pwd command in bin/crunz right before the call to the lavary/crunz script. It prints out the current working directory (.../public_html). I also added a line at the top of the lavary/crunz script echo getcwd(). It prints out .../public_html/vendor/lavary/crunz. This suggests cron is changing the directory when the script is run. I don't want to waste anymore of anybodies time if this is a server config problem but it starting to look that way.

[eradin]

Ok, I figured out the problem. I am documenting the reasons here in hope I save some poor sole some valuable time and frustration. It should also improve the out of the box experience for newbies. This is really a great tool and it should work right out of the box.

I ran several tests independent of crunz. Apparently, getcwd(), which crunz is using, returns different results based on the php interpreter being used. In my case, the CLI was using cli and cron was using cgi-fcgi. The cli version will return the working directory as set in the shell. Cron (and the web server) will return the path of the invoked script. Since my versions are different, I get different results. You can see what cron is using by running php -v > phpver.log as a cron job (run the same from the shell and compare).

Now, this means crunz won't work properly in all environments (I found a DRUPAL post which states the same issue with their product). Maybe, setting an environment variable somewhere to the cwd prior to the lavary/crunz script being called and then reset the cwd at the top of that script. I suppose there are other ways too but it should be fixed since crunz is primarily a cron solution. I gave my workaround in a prior post using symbolic links. I really didn't want to change the code to make it upgrade safe.

Thanks for all who chimed in with suggestions.

[PabloKowalczyk]

@eradin did passing absolute path to PHP binary solves issue also?

[eradin]

I'm not sure what you mean. I did try to replace the shebang with /usr/bin/php instead of the env php and that worked fine. Cron has it's own environment and path so using env is unpredictable. But hard coding /usr/bin/php is also unpredictable. The environment variable cwd is correct before the call to the second lavary/crunz. I think if you copy that to say CRUNZ_ROOT before the call to lavary/crunz and then pick it up in lavary/crunz, you can't go wrong in any case. I was going to try it out and if it worked post a pull request but I'm a little busy this week and can't get to it for several days.

[PabloKowalczyk]

I mean crontab entry like * * * * * cd /project/path && /absolute/path/to/bin/php vendor/bin/crunz s:r. With this entry you specify which PHP binary will be used. If this helps you I will change documentation.

[eradin]

No. That's because the error occurs in the lavary/crunz file (the vendor/bin/crunz call is fine). The shebang says env php so it will ignore your path entry and go with the environment. That is running cgi-fcgi. In a shared environment, I have no control over the cron environment.

[PabloKowalczyk]

Shebang will be used only if you call vendor/bin/crunz directly, if you specify PHP binary (/absolute/path/to/bin/php vendor/bin/crunz), specified binary will be used.

[eradin]

Maybe I'm not getting something here. vendor/bin/crunz is not a php file, it's a shell script. Running it through php just dumps it out to stdout. That file runs just fine as a shell script. It then calls lavary/crunz and that is a php file. It's also where the trouble begins because it's not using the sh interpreter (in my case). The standard php interpreter assumes the cwd is the path to the current running script. We need to force the php script to use the sh interpreter or you will have the same issues I have. What am I missing here?

[eradin]

Here, this is the solution:

vendor/bin/crunz line 14 insert:

export CRUNZ_ROOT=$PWD

lavary/crunz line 16 insert:

$ROOT = getenv('CRUNZ_ROOT');
if ($ROOT !== FALSE) {
chdir($ROOT);
}

[PabloKowalczyk]

I'm wondering why your vendor/bin/crunz is shell script? On Ubuntu and in Alpine container it is symlink to PHP file, are you on Windows? Maybe your Composer is very old?

I would like to understand reason of your issue, you wrote:

In my case, the CLI was using cli and cron was using cgi-fcgi

Does it mean that Cron was using FPM runtime? How can i reproduce your environment with Docker?

IMO the best solution will be to add new option to schedule:run, or whole Crunz, --cwd=/cwd/path, this will definitely reduce confusion and will be runtime independent. WDYT?

[eradin]

My dev system is windows and I propagate to a linux server. My composer version is 1.9 (8/2019) so it's pretty current. bin/crunz is most definitely a sh script and it does run on linux. I've never had a problem with propagating any php library from windows to linux.

I'm not sure you can reproduce it. My cpanel hosting has had other peculiar setups that don't seem to appear on my vps servers. I'm really alerting you that these kinds of things do occur and you should find a way to work around it. Instead of the CD ... in the cron command, I think having a command line arg is a great idea as long as you can search that path for the yml file and task directory. I implemented the environment variable approach from yesterday and it also works great and is transparent. Regarding your FPM question, If you php -v from the linux command line, it's states (cli). If you do the same from cron (I just added a job of php -v > phpver.log) and it reported cgi-fcgi which works differently than (sh). sh takes the current cwd while cgi changes the cwd to DIR.

Now, why do I have a sh script instead of a php script? Either way, it should work and as I've demonstrated, it can work transparently with a few lines of script. And that solution doesn't require any documentation changes.

Let me know if you have other questions.

[eradin]

Btw, here is what I have after composer install on windows.

bin/crunz.bat

@ECHO OFF
setlocal DISABLEDELAYEDEXPANSION
SET BIN_TARGET=%~dp0/../lavary/crunz/crunz
php "%BIN_TARGET%" %*

bin/crunz

#!/usr/bin/env sh

dir=$(cd "${0%[/\\]*}" > /dev/null; cd "../lavary/crunz" && pwd)

if [ -d /proc/cygdrive ]; then
    case $(which php) in
        $(readlink -n /proc/cygdrive)/*)
            # We are in Cygwin using Windows php, so the path must be translated
            dir=$(cygpath -m "$dir");
            ;;
    esac
fi

"${dir}/crunz" "$@"

[PabloKowalczyk]

Did you deploy your application by FTP/SCP/rsync/etc?

Your solution with modifying vendor/bin/crunz (which is binary file, not PHP file) is not portable at all, better solution for long term is --cwd option. Would you like to provide PR?

[eradin]

Yes I use rsync. I don't understand what you mean the vendor/bin/cruz file is binary. It's a text shell script. At least my copy is. Anyway, I have it working with my patch but I would like a more permanent solution. If you think --cwd will do it, great.

[eradin]

I just tried to install crunz on the linux server and it is a very different setup than on windows. Perhaps there is a mismatch on how I am using crunz and the intended use. I have it as part of my application distro and I also check it into git. All of these symbolic links kind of break that development paradigm.

Was the intended use to make it a server deployment dependency? I suppose I could work that into my deployment script.

[PabloKowalczyk]

It is always better to install dependencies on target OS. When Crunz is installed on Linux, Composer creates symlink to Crunz main file and you can use /path/to/your/php-bin vendor/bin/crunz, of course course this is just a simplification. Solution will be a --cwd option.