libiconv missing from uwsgi dependencies

[lsetiawan]

I get an error when trying to deploy wofpy via uwsgi.

$ uwsgi --socket 0.0.0.0:8080 --protocol=http -w wsgi
uwsgi: error while loading shared libraries: libiconv.so.2: cannot open shared object file: No such file or directory

To solve this, I had to do conda install libiconv. Seems like this dependency is missing from uwsgi, should it be included in wofpy?

[lsetiawan]

@ocefpaf Do you know if libiconv is included in uwsgi conda feedstock?

[ocefpaf]

It is the dependency of a dependency (libxml2 I guess) .

The problem is probably a bad mix of packages from defaults and conda-forge. I believe that defaults does not add libiconv to their stack.

Maybe conda-forge/uwsgi-feedstock#10 will fix it. (But things will only get stable when conda-forge moves to conda-build 3.)

[lsetiawan]

Thanks for figuring things out 😄

[emiliom]

Thanks @ocefpaf. Does this mean that it's safer for @lsetiawan to add that dependency explicitly in WOFpy requirements until conda-forge moves to conda-build 3? Or that we should wait (while crossing our fingers) to see if conda-forge/uwsgi-feedstock#10 will take care of it?

[ocefpaf]

Does this mean that it's safer for @lsetiawan to add that dependency explicitly in WOFpy requirements until conda-forge moves to conda-build 3?

In a way, yes :-( b/c even with conda-forge/uwsgi-feedstock#10, if defaults updates their version too, there will still be conda solver related problems like that and we cannot predict when defaults will updated.

[emiliom]

Ok, thanks. So be it.

[lsetiawan]

I tried deploying WOFpy again with uwsgi and I am still getting a libiconv error similar to above.

[ocefpaf]

I tried deploying WOFpy again with uwsgi and I am still getting a libiconv error similar to above.

That means you are using the conda-forge channel for deploying, right?
For Travis-CI we split the coding standards tests, that uses conda-forge to get those flake8 plugins, to avoid the conflict with the odm2 and defaults channels.

If you are using conda-forge I guess that the best solution would be to re-build the odm2 channel inside the conda-forge context. If you are not using conda-forge you should be able to use only defaults and odm2, but not the 3 of them.

The reason for this incompatibility is b/c the the odm2 channel is built against defaults packages.

PS: can you send me the details on how you are creating the deployment env?

[lsetiawan]

I did this:

1. conda create -n wofpytest -c odm2 python=2.7 --file requirements.txt -- file requirements-dev.txt

[ocefpaf]

What is your .condarc?

[lsetiawan]

I do not have one.

[ocefpaf]

Can you send me a conda list for that env? If you have conda-forge packages in there than it is related to the issue I mentioned above, if not than it is an Anaconda issue and we should report it upstream.

PS: if you do have conda-forge packages you somehow have a .condarc somewhere 😄

[lsetiawan]

I don't have any conda-forge packages within this environment. Only defaults and odm2. Should I be using conda-forge also? I thought I can't use all 3?

Conda List

# packages in environment at /home/lsetiawan/Miniconda/envs/wofpytest:
#
# Name                    Version                   Build  Channel
asn1crypto                0.24.0                   py27_0  
attrs                     17.4.0                   py27_0  
automat                   0.6.0            py27hfed452d_0  
ca-certificates           2017.08.26           h1d4fec5_0  
certifi                   2018.1.18                py27_0  
cffi                      1.11.4           py27h9745a5d_0  
chardet                   3.0.4            py27hfa10054_1  
click                     6.7              py27h4225b90_0  
configparser              3.5.0            py27h5117587_0  
constantly                15.1.0                   py27_0  
cryptography              2.1.4            py27h6697b16_0  
django                    1.11.8           py27hd476221_0  
docopt                    0.6.2                    py27_0  
enum34                    1.1.6            py27h99a27e9_1  
flask                     0.12.2           py27h6d5c1cd_0  
funcsigs                  1.0.2            py27h83f16ab_0  
geoalchemy-odm2           0.7.4                    py27_1    odm2
geos                      3.6.2                heeff764_2  
hyperlink                 17.1.1                   py27_0  
icu                       58.2                 h9c2bf20_1  
idna                      2.6              py27h5722d68_1  
incremental               17.5.0           py27h1923969_0  
intel-openmp              2018.0.0             hc7b2577_8  
ipaddress                 1.0.19                   py27_0  
itsdangerous              0.24             py27hb8295c1_1  
jinja2                    2.10             py27h4114e70_0  
krb5                      1.14.2               hcdc1b81_6  
libedit                   3.1                  heed3624_0  
libffi                    3.2.1                hd88cf55_4  
libgcc-ng                 7.2.0                h7cc24e2_2  
libgfortran-ng            7.2.0                h9f7466a_2  
libpq                     9.6.6                h4e02ad2_0  
libsodium                 1.0.15               hf101ebd_0  
libstdcxx-ng              7.2.0                h7a57d05_2  
libxml2                   2.9.7                h26e45fe_0  
libxslt                   1.1.32               h1312cb7_0  
lxml                      4.1.1            py27hf71bdeb_1  
markupsafe                1.0              py27h97b2822_1  
mkl                       2018.0.1             h19d6760_4  
msgpack-python            0.5.1            py27h6bb024c_0  
ncurses                   6.0                  h9df7e31_2  
nose                      1.3.7            py27heec2199_2  
numpy                     1.9.3            py27h7e35acb_3  
odm2api                   0.7.0                    py27_1    odm2
openssl                   1.0.2n               hb7f436b_0  
pandas                    0.21.1           py27h38cdd7d_0  
pastedeploy               1.5.2                    py27_1  
pcre                      8.41                 hc27e229_1  
pip                       9.0.1            py27ha730c48_4  
pluggy                    0.6.0            py27h1f4f128_0  
psycopg2                  2.7.3.2                  py27_0    odm2
py                        1.5.2            py27h203d672_0  
pycodestyle               2.3.1            py27h904819d_0  
pycparser                 2.18             py27hefa08c5_1  
pymysql                   0.7.11           py27ha7233d9_0  
pyodbc                    4.0.22           py27hf484d3e_0  
pyopenssl                 17.5.0           py27hcee3be0_0  
pyparsing                 2.2.0            py27hf1513f8_1  
pyramid                   1.5.7                    py27_0  
pysocks                   1.6.7            py27he2db6d2_1  
pytest                    3.3.2                    py27_0  
python                    2.7.14              h1571d57_29  
python-dateutil           2.6.0                    py27_1    odm2
pytz                      2017.3           py27h001bace_0  
pyyaml                    3.12             py27h2d70dd7_1  
pyzmq                     16.0.3           py27hc579512_0  
readline                  7.0                  ha6073c6_4  
repoze.lru                0.6                      py27_0  
requests                  2.18.4           py27hc5b0589_1  
setuptools                38.4.0                   py27_0  
shapely                   1.5.15               np19py27_0    odm2
simplejson                3.12.0           py27h264bab0_0  
six                       1.11.0           py27h5f960f1_1  
spyne                     2.12.14                  py27_2    odm2
sqlalchemy                1.2.1                    py27_0    odm2
sqlite                    3.22.0               h1bed415_0  
suds-jurko                0.6                      py27_1    odm2
tk                        8.6.7                hc745277_3  
translationstring         1.3                      py27_0  
twisted                   17.5.0                   py27_0  
unixodbc                  2.3.4                hc36303a_1  
urllib3                   1.22             py27ha55213b_0  
uwsgi                     2.0.15                   py27_0    odm2
venusian                  1.0                      py27_0  
webob                     1.4.1                    py27_0  
werkzeug                  0.14.1                   py27_0  
wheel                     0.30.0           py27h2bc6bb2_1  
WOFpy                     2.3.0+9.g2d9c633           <pip>
xz                        5.2.3                h55aa19d_2  
yaml                      0.1.7                had09818_2  
zeromq                    4.2.2                hbedb6e5_2  
zlib                      1.2.11               ha838bed_2  
zope                      1.0                      py27_0  
zope.deprecation          4.1.2                    py27_1  
zope.interface            4.1.2                    py27_1 

Details

[ocefpaf]

This is what I feared 😒

Let me go over all the packages in odm2 to be sure where is contamination is coming from but I think the issue is in defaults.