Re: [Fuego] Installed plugins not listed under ‘Plu gin Manager’ -> ‘Installed’ page

Dhinakar,

I'm sorry that you are having problems with the Jenkins plugins.
This has been a perennial problem for Fuego, trying to keep up with the constant
churn as the versions of Jenkins and its plugins keeps changing.

Here is some information and some tips, in the hopes that it will be useful for you.

When Fuego installs, it uses a hardcoded Jenkins version.
In previous versions of Fuego, we hardcoded frozen versions of the Jenkins plugins,
so that when the docker container was created, it didn't pick up arbitrary plugin
versions, that might not be compatible with the version of Jenkins used in 
the Fuego docker image.

However, the last time we did a Jenkins version upgrade was in Dec 1, 2020.
At that time, for reasons unknown, we changed the Dockerfile so that the
plugin versions could float (that is, the latest version of each Jenkins plugin
would be used).  This code has since been refactored to make native installs
easier, but the use of non-frozen versions of plugins still remains.

In any event, I have an old docker container from December of 2020, that has
the original versions of plugins that were compatible with Jenkins 2.249.3
(which was the version of Jenkins used in the 1.5.0 release of Fuego).

Here are the version of plugins  in that container:
ant:1.11
antisamy-markup-formatter:2.1
bootstrap4-api:4.5.3-1
bouncycastle-api:2.18
checks-api:1.1.1
command-launcher:1.5
description-setter:1.10
display-url-api:2.3.4
echarts-api:4.9.0-2
external-monitor-job:1.7
flot:1.0-SNAPSHOT (private-07/21/2017 10:45-dsl)
font-awesome-api:5.15.1-1
greenballs:1.15
icon-shim:2.0.3
jackson2-api:2.12.0
javadoc:1.6
jdk-tool:1.4
jquery3-api:3.5.1-2
junit:1.45
ldap:1.11
mailer:1.32.1
matrix-auth:2.6.4
matrix-project:1.18
pam-auth:1.6
pegdown-formatter:1.3
plugin-util-api:1.5.0
popper-api:1.16.0-7
scm-api:2.6.4
script-security:1.75
snakeyaml-api:1.27.0
structs:1.20
trilead-api:1.0.4
windows-slaves:1.7
workflow-api:2.40
workflow-step-api:2.23

the Fuego install system uses an external script to install plugins (rather than doing so inside
Jenkins itself).  This script is found inside the fuego docker container at: /usr/local/bin/install-plugins.sh

To use it, you specify a list of plugins to install, on the command line.  You can include a version number
for a package by including the version number following the package name and a colon (e.g. structs:1.20)

It sounds like some of your plugin operations have upgraded your existing plugins, so that now there
are dependency problems with the collection of plugins on your system.

There are 2 ways to proceed:  1) Stay with version 2.249.3 of Jenkins, and try to revert your plugins to ones
that will work with that version of Jenkins, or 2) migrate to a newer version of Jenkins.

Option 2 is, IMHO, fraught with peril. But, depending on the extra modules you want to install and your
sensitivity to security issues, it may be the best route forward.  If the jira plugins you want to install
rely on newer modules which themselves rely on a newer Jenkins version, then in order to install
those you may have no choice but to update the version of Jenkins.

Option 1 also entails some danger (as I have not tried the instructions I am about to provide), but 
it *should* work.  Try uninstalling the plugins that have versions that don't match the ones in
the above list (that are known to be compatible with Jenkins 2.249.3).  Then use install-plugins.sh
to re-install specific versions of the modules that match the list above.

In order to run install-plugins.sh, you need to define two environment variables, to indicate
the Jenkins upload center and the Jenkins plugin directory.  These are names JENKINS_UC
and REF, respectively.

$ export JENKINS_UC=https://updates.jenkins.io
$ export REF=/var/lib/jenkins/plugins
$ ./install-plugins.sh <plugin1> <plugin2> ...

To test the script, and get a list of currently installed plugins, you can specify to install
a non-existent plugin.  Here's what I did, and the output I got back from it:
Note that I get some curl certificate errors, but I think those can be ignored for the purposes
of this operation.

--------

root@timdesk:/usr/local/bin# export JENKINS_UC=https://updates.jenkins.io
root@timdesk:/usr/local/bin# export REF=/var/lib/jenkins/plugins
root@timdesk:/usr/local/bin# ./install-plugins.sh foo
Creating initial locks...
Analyzing war...
Registering preinstalled plugins...
Downloading plugins...
Downloading plugin: foo from https://updates.jenkins.io/download/plugins/foo/latest/foo.hpi
curl: (60) SSL certificate problem: certificate has expired
More details here: https://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
 of Certificate Authority (CA) public keys (CA certs). If the default
 bundle file isn't adequate, you can specify an alternate file
 using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
 the bundle, the certificate verification probably failed due to a
 problem with the certificate (it might be expired, or the name might
 not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
 the -k (or --insecure) option.
23:57:40 Failure (60) Retrying in 1 seconds...
curl: (60) SSL certificate problem: certificate has expired
More details here: https://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
 of Certificate Authority (CA) public keys (CA certs). If the default
 bundle file isn't adequate, you can specify an alternate file
 using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
 the bundle, the certificate verification probably failed due to a
 problem with the certificate (it might be expired, or the name might
 not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
 the -k (or --insecure) option.
23:57:42 Failure (60) Retrying in 1 seconds...
curl: (60) SSL certificate problem: certificate has expired
More details here: https://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
 of Certificate Authority (CA) public keys (CA certs). If the default
 bundle file isn't adequate, you can specify an alternate file
 using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
 the bundle, the certificate verification probably failed due to a
 problem with the certificate (it might be expired, or the name might
 not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
 the -k (or --insecure) option.
23:57:43 Failure (60) Retrying in 1 seconds...
23:57:44 Failed in the last attempt (curl -sSfL --connect-timeout 20 --retry 3 --retry-delay 0 --retry-max-time 60 https://updates.jenkins.io/download/plugins/foo/latest/foo.hpi -o /var/lib/jenkins/plugins/foo.jpi)
./install-plugins.sh: line 63: $3: unbound variable

WAR bundled plugins:


Installed plugins:
ant:1.11
antisamy-markup-formatter:2.1
bootstrap4-api:4.5.3-1
bouncycastle-api:2.18
checks-api:1.1.1
command-launcher:1.5
description-setter:1.10
display-url-api:2.3.4
echarts-api:4.9.0-2
external-monitor-job:1.7
flot:1.0-SNAPSHOT (private-07/21/2017 10:45-dsl)
font-awesome-api:5.15.1-1
greenballs:1.15
icon-shim:2.0.3
jackson2-api:2.12.0
javadoc:1.6
jdk-tool:1.4
jquery3-api:3.5.1-2
junit:1.45
ldap:1.11
mailer:1.32.1
matrix-auth:2.6.4
matrix-project:1.18
pam-auth:1.6
pegdown-formatter:1.3
plugin-util-api:1.5.0
popper-api:1.16.0-7
scm-api:2.6.4
script-security:1.75
snakeyaml-api:1.27.0
structs:1.20
trilead-api:1.0.4
windows-slaves:1.7
workflow-api:2.40
workflow-step-api:2.23
Cleaning up locks

-----------------

Please try either option 1 or 2 and let me know what happens.

By the way, you can shutdown Jenkins inside the docker container, and restart it, by doing this,
inside the container, as root.
$ service jenkins stop
$ service jenkins start

Doing this might clear up the issue with plugins being present in the installation directory
but not registered with the system.  You probably want to do this after installing plugins
from the docker container command line.

You might also try to reset Jenkins from inside the web interface by going to "manage Jenkins",
"Reload Configuration from Disk".

I hope this helps.  Let me know what you try, and what the eventual outcome is.

Best of luck!
 -- Tim

P.S. If you decide to upgrade the version of Jenkins, make sure you check the Jenkins
config.xml for items that are needed by Fuego for proper Jenkins operations.
Sometimes, newer Jenkins versions will change the items in the config.xml file
in a way that is not backwards compatible.  The config.xml that Fuego uses when
it installs Jenkins into the docker container originally is in: fuego/frontend-install/config.xml


> -----Original Message-----
> From: Dhinakar Kalyanasundaram/FDS SW /SSIR/Staff Engineer/삼성전자 <[email protected]>
> 
> Dear Fuego users,
> 
> 
> 
> I am using Fuego VERSION=1.5.0 and Jenkins ver. 2.164.2
> 
> 
> 
> I was trying to install a few Jira plugins to interface from Jenkins.
> 
> After a while I noticed the below warning message (in many of my plugins) and many of my already installed & setup plugins like ‘Ant’,
> ‘Structs’, ‘Sidebar Link’, ‘Gerrit Trigger’ etc. were missing even though their folders and *.jpi files were still available under
> ‘/var/lib/jenkins/plugins’ directory inside docker container.
> 
> 
> 
> Warning: This plugin requires dependent plugins that require Jenkins 2.249.1 or newer. Jenkins will refuse to load the dependent plugins
> requiring a newer version of Jenkins, and in turn loading this plugin will fail.
> 
> 
> 
> Now, I see many of my above plugin’s next version listed under ‘Plugin Manager’ -> ‘Available’ with the above warning message.
> 
> For e.g. Structs 1.20 version is already installed but not listed under ‘Plugin Manager’ -> ‘Installed’ page but instead ‘Plugin Manager’ ->
> ‘Available’ is showing the next version of ‘Structs’ with the warning shown above (in red italics).
> 
> 
> 
> I tried removing the newly installed plugins and restart Jenkins & docker container multiple times but still the issue was not solved.
> 
> Any suggestions to fix this?
> 
> Since I have lot of jobs configured with plugins like ‘Sidebar Link’, ‘Post Build Task’ etc. I cannot reinstall or upgrade fuego.
> 
> Thanks in advance.
> 
> 
> 
> Regards,
> 
> Dhinakar
> 
> 
> 
> 
> 
> 
> 
> 
> 
> <http://ext.samsung.net/mail/ext/v1/external/status/update?userid=dhinakar.k&do=bWFpbElEPTIwMjIwMjE3MTA1NjI3ZXBjYXM1cDFlMjR
> hZDlmZGRjNTA1ZDAyNmRiZmU4NjQ4ZmU4YTVhYSZyZWNpcGllbnRBZGRyZXNzPVRpbS5CaXJkQHNvbnkuY29t>