Make plugin compatible with hirak/prestissimo #21

mcaskill · 2020-09-06T14:44:02Z

Checklist:

I've read the Contributing page.
I've created an issue and referenced it here.
My code is tested.
My code follows the WordPress code style.
My code has proper inline documentation.

Description

When using this Composer plugin in conjunction with hirak/prestissimo, the download of premium plugins fails because the custom RemoteFileSystem is overwritten by the latter.

How has this been tested?

I've tested this on a client project that uses Advanced Custom Fields Pro, Gravity Forms, and Polylang Pro.

Types of changes

As both are running with the default priority (0), setting this one to -1 resolves the issue (this plugin's event handler would be executed after Prestissimo, effectively replacing their custom CurlRemoteFileSystem).

Similar:

Resolves #1, #14

Alternatively, we could change the priority for this package to execute after Prestissimo and take advantage of the parallel downloads but we would have to perform some kind of trickery where we supply the real download URL to Prestissimo (via $event->getProcessedUrl()) but ensure that the package's dist.url does not use that URL.

References: - PhilippBaschke/acf-pro-installer#20 - ffraenz/private-composer-installer@fcc2195 Resolves #1, junaidbhura#14

junaidbhura · 2020-09-11T06:37:13Z

Thanks @mcaskill looks good to me!

Twansparant · 2020-09-11T07:17:16Z

I cheered too soon so it seems...
I updated to composer-wp-pro-plugins 1.0.9 but I keep on getting the following error when deploying:

Installing junaidbhura/gravityformsmailchimp (4.8): Downloading (100%) Failed
to execute (9) unzip -qq
'web/app/plugins/gravityformsmailchimp//c19997d111893af0a5baad7331c3a316' -d 
'/srv/www/mydomain/releases/20200911092303/vendor/composer/cce734
44' [web/app/plugins/gravityformsmailchimp//c19997d111893af0a5baad7331c3a316]
End-of-central-directory signature not found. Either this file is not a
zipfile, or it constitutes one disk of a multi-part archive. In the latter
case the central directory and zipfile comment will be found on the last
disk(s) of this archive. unzip: cannot find zipfile directory in one of
web/app/plugins/gravityformsmailchimp//c19997d111893af0a5baad7331c3a316 or
web/app/plugins/gravityformsmailchimp//c19997d111893af0a5baad7331c3a316.zip,
and cannot find
web/app/plugins/gravityformsmailchimp//c19997d111893af0a5baad7331c3a316.ZIP,
period. The archive may contain identical file names with different
capitalization (which fails on case insensitive filesystems) Unzip with unzip
command failed, falling back to ZipArchive class Invalid zip file,
retrying...

Locally everything works?

mcaskill · 2020-09-11T13:01:15Z

Hi @Twansparant,

That is puzzling. I admit my testing was limited to my computer only.

Having spent quite some time these past two weeks reading and playing with this Composer plugin, I'm pretty certain the double slash is not coming from it. There's no manipulation of the dist.url, cache key (the MD5 hash of the dist.url that is used as the file name), nor of the path from the cache or to the installation path.

What version of hirak/prestissimo and composer/installers are you using?
What version of Composer are you running locally and on your production server?
What OS are you running locally and on your production server?
What is the exact file name that is stored in Composer's cache for the failing files?

mcaskill · 2020-09-12T17:09:43Z

Hi @Twansparant

I using Composer's verbose output while working on another pull request and I noticed the double slash (without using Prestissimo). On my end, Composer still manages to properly unzip the file.

I looked around to see where this might be coming from and I believe it's coming from Composer's FileDownloader::getFileName() forcing a slash between the install path and the basename:

protected function getFileName(PackageInterface $package, $path)
{
    return $path.'/'.pathinfo(parse_url($package->getDistUrl(), PHP_URL_PATH), PATHINFO_BASENAME);
}

In combination with WordPressInstaller trailing slashes:

protected $locations = array(
    'plugin'    => 'wp-content/plugins/{$name}/',
    'theme'     => 'wp-content/themes/{$name}/',
    'muplugin'  => 'wp-content/mu-plugins/{$name}/',
    'dropin'    => 'wp-content/{$name}/',
);

I recommend you create an issue on composer/composer or composer/installers.

Twansparant · 2020-09-14T09:49:08Z

Hi @mcaskill,

No worries, I'm just really keen on getting this to work in our Trellis setups!

What version of hirak/prestissimo and composer/installers are you using?

"junaidbhura/composer-wp-pro-plugins": "^1.0.9"
"composer/installers": "^1.8"
"vlucas/phpdotenv": "^4.1.0"
"oscarotero/env": "^2.1"

For hirak/prestissimo I don't know to be honest, it doesn't show up on composer global show -i locally or on the remote?

2. What version of Composer are you running locally and on your production server?

Local: Composer version 1.10.10
Production: Composer version 1.10.8

3. What OS are you running locally and on your production server?

Local: Ubuntu 18.04.4 LTS (GNU/Linux 4.15.0-91-generic x86_64)
Production: Ubuntu 18.04.3 LTS (GNU/Linux 4.15.0-112-generic x86_64)

4. What is the exact file name that is stored in Composer's cache for the failing files?

For example for GravityForms itself:

Loading
from cache - Installing junaidbhura/gravityforms (2.4.20.4): Downloading
(100%) Failed to execute (9) unzip -qq
'web/app/plugins/gravityforms//ee5931097fc2f11b7edde5b0dd4d8da6'

When I list the composer cache on the production server:

ls -la ~/.cache/composer/files/junaidbhura/composer-wp-pro-plugins

it has one file:

total 20
drwxr-xr-x 2 web www-data  4096 Sep 14 09:39 .
drwxr-xr-x 4 web www-data  4096 Sep 14 09:39 ..
-rw-r--r-- 1 web www-data 10430 Sep 14 09:39 8332a39999c8133326eda8436148aa33585ba307.zip

with a creation date of the last deploy

ls -la ~/.cache/composer/files/junaidbhura/gravityforms

is empty.

Twansparant · 2020-09-14T10:10:39Z

I looked around to see where this might be coming from and I believe it's coming from Composer's FileDownloader::getFileName() forcing a slash between the install path and the basename:

I override these wordpress installer-paths in my composer.json. If I remove the trailing slash for the plugins path:

  "extra": {
    "installer-paths": {
      "web/app/mu-plugins/{$name}/": ["type:wordpress-muplugin", "booreiland/responsive-pics"],
      "web/app/plugins/{$name}": ["type:wordpress-plugin"],
      "web/app/themes/{$name}/": ["type:wordpress-theme"]
    },
    "wordpress-install-dir": "web/wp"
  },

And deploy again, it has the same error, but without the double trailing slashes in the path:

Installing junaidbhura/gravityforms (2.4.20.4): Downloading (100%) Failed to
execute (9) unzip -qq
'web/app/plugins/gravityforms/69e183a865acac45d098dddb1eae8903' -d '/srv/www/
mydomain.com/releases/20200914100319/vendor/composer/94a4f8d6'
[web/app/plugins/gravityforms/69e183a865acac45d098dddb1eae8903] End-of-
central-directory signature not found. Either this file is not a zipfile, or
it constitutes one disk of a multi-part archive. In the latter case the
central directory and zipfile comment will be found on the last disk(s) of
this archive. unzip: cannot find zipfile directory in one of
web/app/plugins/gravityforms/69e183a865acac45d098dddb1eae8903 or
web/app/plugins/gravityforms/69e183a865acac45d098dddb1eae8903.zip, and cannot
find web/app/plugins/gravityforms/69e183a865acac45d098dddb1eae8903.ZIP,
period. The archive may contain identical file names with different
capitalization (which fails on case insensitive filesystems) Unzip with unzip
command failed, falling back to ZipArchive class Invalid zip file,
retrying...

So it's probably not the extra slash causing this?
The thing is, when I ssh into my production server and run composer install in the failed deploy folder, there are NO errors? Also, the first deploy after that, does succeed?

junaidbhura · 2020-10-02T07:29:24Z

Hey @Twansparant can you try removing --no-scripts when you install?

Twansparant · 2020-10-02T07:56:58Z

Hey @Twansparant can you try removing --no-scripts when you install?

Hi @junaidbhura, I just merged this PR, and set the composer_no_scripts to false:

# Most scripts are used in development instead of remote servers. Use with caution.
composer_no_scripts: false

and deployed again, and the errors are gone!
I'm gonna try it in a few other projects to verify if this was the culprit.
Thanks!

Twansparant · 2020-10-02T09:04:09Z

Hey @Twansparant can you try removing --no-scripts when you install?

This seems to be working now without errors!
But somehow, it doesn't install the correct plugin versions defined in the composer.json or composer.lock:

    {
      "type": "package",
      "package": {
        "name": "junaidbhura/gravityforms",
        "version": "2.4.20.5",
        "type": "wordpress-plugin",
        "dist": {
          "type": "zip",
          "url": "https://www.gravityforms.com"
        },
        "require": {
          "junaidbhura/composer-wp-pro-plugins": "*"
        }
      }
    },

And on the deploy it installed version: 2.4.17.14?
Even when I force the latest composer-wp-pro-plugins version 1.0.10 that prevents Composer caching:

"junaidbhura/composer-wp-pro-plugins": "1.0.10"

junaidbhura · 2020-10-02T09:07:14Z

Hey @Twansparant great to hear that it works now! There's a known issue that Gravity Forms will always download the latest version: https://github.com/junaidbhura/composer-wp-pro-plugins#composer-installer-for-pro-wordpress-plugins

Twansparant · 2020-10-02T09:09:52Z

There's a known issue that Gravity Forms will always download the latest version

I wish it did download the latest version! I defined the latest version, but I get a version from months ago?

junaidbhura · 2020-10-02T09:14:27Z

@Twansparant did you do a composer update recently?

Twansparant · 2020-10-02T09:16:20Z

Yes, before deploying I always make sure my .lock is up to date.
The lock file also has the 2.4.20.5 version in it?

junaidbhura · 2020-10-02T09:17:07Z

And do you get the same version locally?

Twansparant · 2020-10-02T09:17:58Z

No, locally the correct 2.4.20.5 version is installed?
I'm looking at any other ansible composer parameters, but I don't see any relevant ones?

Twansparant · 2020-10-02T09:39:22Z

Ok, I updated vlucas/phpdotenv to v4.1, did a clean composer install deployed and it installed the correct versions now!

junaidbhura · 2020-10-02T09:43:59Z

Awesome!

mcaskill · 2020-10-02T14:48:30Z

In the lock file, the entry for Gravity Forms should look like this for 2.4.20.5:

{
    "name": "junaidbhura/gravityforms",
    "version": "2.4.20.5",
    "dist": {
        "type": "zip",
        "url": "https://www.gravityforms.com/#a42d513142b5f342c5237ac16a2a8279ed5270bb"
    },
    "require": {
        "junaidbhura/composer-wp-pro-plugins": "*"
    },
    "type": "wordpress-plugin"
}

I've noticed the issue when updating to composer-wp-pro-plugins v1.0.10 in #7:

[…] Sometimes the pro WordPress plugins will update before the Composer plugin updates to v1.0.10, resulting in a continuance of the caching issue from before.

The solution I've found to this is either deleting the the WordPress plugins to force Composer to re-evaluate the packages and download a fresh copies using the correct dist URLs.

Alternatively, you can update the composer.lock manually by providing the WordPress plugins with the correctly hashed dist URL.

Make plugin compatible with hirak/prestissimo

70c2da2

References: - PhilippBaschke/acf-pro-installer#20 - ffraenz/private-composer-installer@fcc2195 Resolves #1, junaidbhura#14

mcaskill force-pushed the bugfix/prestissimo-compat branch from 8b809c9 to 70c2da2 Compare September 6, 2020 14:45

junaidbhura merged commit 89f27ef into junaidbhura:master Sep 11, 2020

mcaskill deleted the bugfix/prestissimo-compat branch September 28, 2020 18:17

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make plugin compatible with hirak/prestissimo #21

Make plugin compatible with hirak/prestissimo #21

mcaskill commented Sep 6, 2020

junaidbhura commented Sep 11, 2020

Twansparant commented Sep 11, 2020 •

edited

Loading

mcaskill commented Sep 11, 2020

mcaskill commented Sep 12, 2020

Twansparant commented Sep 14, 2020 •

edited

Loading

Twansparant commented Sep 14, 2020 •

edited

Loading

junaidbhura commented Oct 2, 2020

Twansparant commented Oct 2, 2020 •

edited

Loading

Twansparant commented Oct 2, 2020 •

edited

Loading

junaidbhura commented Oct 2, 2020

Twansparant commented Oct 2, 2020

junaidbhura commented Oct 2, 2020

Twansparant commented Oct 2, 2020 •

edited

Loading

junaidbhura commented Oct 2, 2020

Twansparant commented Oct 2, 2020 •

edited

Loading

Twansparant commented Oct 2, 2020

junaidbhura commented Oct 2, 2020

mcaskill commented Oct 2, 2020 •

edited

Loading

Make plugin compatible with hirak/prestissimo #21

Make plugin compatible with hirak/prestissimo #21

Conversation

mcaskill commented Sep 6, 2020

Checklist:

Description

How has this been tested?

Types of changes

junaidbhura commented Sep 11, 2020

Twansparant commented Sep 11, 2020 • edited Loading

mcaskill commented Sep 11, 2020

mcaskill commented Sep 12, 2020

Twansparant commented Sep 14, 2020 • edited Loading

Twansparant commented Sep 14, 2020 • edited Loading

junaidbhura commented Oct 2, 2020

Twansparant commented Oct 2, 2020 • edited Loading

Twansparant commented Oct 2, 2020 • edited Loading

junaidbhura commented Oct 2, 2020

Twansparant commented Oct 2, 2020

junaidbhura commented Oct 2, 2020

Twansparant commented Oct 2, 2020 • edited Loading

junaidbhura commented Oct 2, 2020

Twansparant commented Oct 2, 2020 • edited Loading

Twansparant commented Oct 2, 2020

junaidbhura commented Oct 2, 2020

mcaskill commented Oct 2, 2020 • edited Loading

Twansparant commented Sep 11, 2020 •

edited

Loading

Twansparant commented Sep 14, 2020 •

edited

Loading

Twansparant commented Sep 14, 2020 •

edited

Loading

Twansparant commented Oct 2, 2020 •

edited

Loading

Twansparant commented Oct 2, 2020 •

edited

Loading

Twansparant commented Oct 2, 2020 •

edited

Loading

Twansparant commented Oct 2, 2020 •

edited

Loading

mcaskill commented Oct 2, 2020 •

edited

Loading