Previous Thread
Next Thread
Print Thread
Page 1 of 2 1 2
testfiles for c35b2 failed
#20573 03/30/09 10:03 AM
Joined: Mar 2009
Posts: 6
S
sassy Offline OP
Forum Member
OP Offline
Forum Member
S
Joined: Mar 2009
Posts: 6
Dear all,

I am having some trouble with the test files for my charmm build (version c35b2).
I build charmm with the gfortran-4.1 compiler and openmpi-1.2.8 using:

./install.com gnu medium M MPISPECIAL x86_64 GFORTRAN

I then downloaded the c35b1_gf_64.tar.gz file which, so I understood, is the right one for my charmm version.

Using the provided test.com, I ran all tests and then running

awk -f compare.awk -v verbose=0 -v tol=0.01 170309.rpt

produced about 50% failures.

Unfortunately, the documentation which is on the CD is completly outdated and one has to hunt for information of how to install and test charmm. So anybody saying now: RTFM will get the reply from me: get me a version of the manual which is acutally up to date. Sorry, but I had to air my frustation here as for a commercial product I found that outrages.

Nonetheless, could anybody point me in the direction what is going on here?

All the best

Jörg

Re: testfiles for c35b2 failed
sassy #20574 03/30/09 04:28 PM
Joined: Sep 2003
Posts: 8,499
rmv Online Content
Forum Member
Online Content
Forum Member
Joined: Sep 2003
Posts: 8,499
You are correct; the documentation is badly out of date, and is among the worst of any such program with the stature of CHARMM.

You should start by comparing single processor test results, w/o MPI

You should probably compare to to c35b2 instead of c35b1

I also suggest upgrading to a more recent gfortran version

Re: testfiles for c35b2 failed
rmv #20575 04/16/09 02:34 PM
Joined: Mar 2009
Posts: 6
S
sassy Offline OP
Forum Member
OP Offline
Forum Member
S
Joined: Mar 2009
Posts: 6
Daer RMV,

thanks for the reply, and sorry for my delayed response.

Updating gfortran sounds like a good idea, however, that would mean updating the whole cluster or building your own gfortran. I do not really want to do either for various reasons

I am doing the build for a single CPU now, see if that goes any better. Unfortunately, I seem to be unable to use the ifort version I have to build the binary. I get problems related to the code during compilation.

As for the test suite, I only found c35b1_gf_64 that which was apparently build on a 64bit platform. I would love to use the c35b2_gf_32 but not only was that build on a 32bit platform, it also uses the 4.2.x gfortran compiler. So here I have changed 2 different parameters.

For a commercial product I find it astonishing not to provdie a reliable test suite.

I will try the singe build and see how it goes.

All the best

Jörg

Re: testfiles for c35b2 failed
sassy #20576 04/16/09 03:33 PM
Joined: Sep 2003
Posts: 8,499
rmv Online Content
Forum Member
Online Content
Forum Member
Joined: Sep 2003
Posts: 8,499
CHARMM is academic software, written and maintained by post-docs, grad students, and working scientists on a strictly volunteer basis; it is an evolving academic research tool, certainly not commercial software. There are commercial entanglements with Accelrys, who re-packages the software as CHARMm, which is one reason an academic license fee is required; CHARMM cannot be made freely available.

There are known issues with gfortran compilers earlier than 4.1.2; caveat emptor. There have been some reports of problems with ifort 10.x as well. In our lab, with a centrally mounted software repository for the cluster, we have multiple compilers available; we recommend gfortran 4.3.x and Intel 11.x for CHARMM. With the central repository, there's no need to update cluster nodes to roll out a new compiler.

Re: testfiles for c35b2 failed
sassy #20577 04/16/09 03:54 PM
Joined: Dec 2005
Posts: 1,535
Forum Member
Offline
Forum Member
Joined: Dec 2005
Posts: 1,535
I can't help noticing that you repeatedly refer to CHARMM as "a commercial product". This is not quite true. CHARMM development is driven by a number of academic groups all over the world. It is a sad truth that we get funded for publishing research papers but not for writing documentation, making our software user-friendly, removing bugs that don't affect the majority of users, or providing support on these very forums. We all do this purely as a community service. If you want to fund a full-time Ph.D. level employee for improving CHARMM's user-friendliness (which, admittedly, is astonishingly low), I'm sure you would make everyone in the CHARMM community very happy. Otherwise, I respectfully urge you to drop the "dissatisfied customer" tone; it won't make you many friends here.

Just for the sake of completeness, I should mention that the company Accelrys does sell a minimally modified version of CHARMM as a commercial product branded CHARMm. If that's what you're using, you could try the "dissatisfied customer" thing with them. Perhaps someone down the line might one day decide to pour some of the profits generated by CHARMm sales into improving CHARMM's user-friendliness.

Edit: oops, rmv beat me to it.

Last edited by Kenno; 04/16/09 03:56 PM.
Re: testfiles for c35b2 failed
Kenno #20578 04/17/09 10:29 AM
Joined: Mar 2009
Posts: 6
S
sassy Offline OP
Forum Member
OP Offline
Forum Member
S
Joined: Mar 2009
Posts: 6
Dear Kenno,

I may quote from the website: "We distribute the source code of the CHARMM program with documentation to individual academic research groups for a $600 licensing fee."
As soon as I pay money for something it is a commercial product in my humble opinion. There are codes which are completly free and they offer an up to date documentation, in some instances even right down to the acutal version you are using, _including_ new features etc.

They are doing that on a volunteer base as well and it works, so I don't see any reason why CHARMM is not able to do that. Writing software you want to distribute also means writing manuals, else you get frustrating people like me. I am not getting paid searching the internet for information of a product I am paying money for.

Sorry for being upfront here. One can do better and I would have thougth that Harvad as one of the IV-leagues could afford somebody to write the documentation of their product they are selling.

All the best

Jörg

Re: testfiles for c35b2 failed
sassy #20579 04/17/09 05:07 PM
Joined: Dec 2005
Posts: 1,535
Forum Member
Offline
Forum Member
Joined: Dec 2005
Posts: 1,535
I for one haven't seen any of that money, and I doubt that anyone on this forum has. Nobody here is very happy with CHARMM's current level of user-friendliness and documentation, but we're powerless to change it. You might be right to complain, but you're complaining to the wrong people. Maybe you should direct your complaints through the same channels as you directed your money...

Re: testfiles for c35b2 failed
Kenno #20580 04/17/09 06:04 PM
Joined: Sep 2003
Posts: 8,499
rmv Online Content
Forum Member
Online Content
Forum Member
Joined: Sep 2003
Posts: 8,499
One reason people like me spend time answering questions in the forum is the knowledge that the documentation is far from good, and people often need some additional guidance. This has led to correcting some errors in the .doc file, and some fixes to install.com among other things.

One problem is that NIH and NSF grants typically don't cover things like improving program documentation, and the principal scientists involved often don't have the resources to support a documentation and packaging improvement project.

Re: testfiles for c35b2 failed
rmv #20581 04/17/09 06:38 PM
Joined: Sep 2003
Posts: 4,794
Likes: 2
Forum Member
Online Content
Forum Member
Joined: Sep 2003
Posts: 4,794
Likes: 2
If we should try to keep things a little bit in perspective:

1. There is really no money at all to speak about (in terms of a large-scale software project) floating around from the academic CHARMM license fee.

2. This thread started with a question about failing testcases. The testcases are being modernized to be self-contained and thus useful for end-users. At present they are best used by developers to make sure that code modifications do not break existing functionality. Many testcases unfortunately use procedures (eg, long simulations) that give slightly different results with different compilers; I think this is what you see. Since CHARMM has very many testcases it will take time to get all this modernized. Anybody is welcome to contribute to this task, and to improving the documentation.

3. User friendliness. Well? The underlying concepts are complicated, and this is software for very complicated tasks (the baseline for this statement is ab initio calculations), and it is a research tool. Very flexible and versatile. It does take time to learn to use it, and to understand what is going on. This is true for all advanced MD software. For "user-friendliness" there are several simple frontends, allowing a more narrow set of tasks to be performed easily: CHARMM-GUI,MMTSB,CHARMMING, Discovery Studio and PipelinePilot, and doubtless many others. A very simple task, running a modernized version of the first protein MD simulation of BPTI (McCammon, Gelin & Karplus (1977 Nature) can be done with this set of commands, and I do not think this is unfriendly at all:
* 10ps dynamics of BPTI in vacuum
*
READ rtf card name top_all22_prot.inp
READ param card name par_all22_prot.inp
READ sequence pdb name bpti.pdb ! bpti pbd file with just BPTI
GENErate bpti
PATCh disu bpti 5 bpti 55
PATCh disu bpti 14 bpti 38
PATCh disu bpti 30 bpti 51
READ coor pdb name bpti.pdb
HBUIld select hydrogen end
shake bonds
OPEN unit 11 write unformatted name bpti.trj
DYNAmics start nstep 10000 timestep 0.001 –
firsttemp 35.0 finaltemp 285.0 teminc 50.0 ihtfrq 200 -
cutnb 14.0 ctofnb 12.0 fshift inbfrq -1 nsavc 100 iuncrd 11

OPEN unit 11 read unformatted name bpti.trj
COOR dyna firstunit 11 nunits 1
WRITe coor pdb name bpti-ave.pdb
* Averaged coordinates. RMS fluctuations in B-factor column
*

I would also like to suggest that starting with simple tasks, and looking carefully at the examples in the Script Archive, is a good way to begin using CHARMM and get started on what admittedly is an uphill learning curve.

Specific questions and suggestions are always welcome.


Lennart Nilsson
Karolinska Institutet
Stockholm, Sweden
Re: testfiles for c35b2 failed
lennart #20582 04/17/09 10:26 PM
Joined: Sep 2003
Posts: 4,794
Likes: 2
Forum Member
Online Content
Forum Member
Joined: Sep 2003
Posts: 4,794
Likes: 2
Added consideration. A move towards increased "user-friendliness" (most users are already very friendly though) would be much helped by a clear definition of "user-friendliness". If we are striving towards something like:
charmm < PDB_ID > manuscript.pdf -jif acceptable_journal_impact_factor
then the default for -jif could probably not be extremely impressive; for more information about impact factors see also Genome Biology 2008, 9:107 (doi:10.1186/gb-2008-9-7-107).


Lennart Nilsson
Karolinska Institutet
Stockholm, Sweden
Re: testfiles for c35b2 failed
lennart #20583 04/17/09 10:57 PM
Joined: Sep 2003
Posts: 8,499
rmv Online Content
Forum Member
Online Content
Forum Member
Joined: Sep 2003
Posts: 8,499
I'm aware of an NMR group that had a publication model close to that

Re: testfiles for c35b2 failed
lennart #20584 04/23/09 11:20 PM
Joined: Dec 2005
Posts: 1,535
Forum Member
Offline
Forum Member
Joined: Dec 2005
Posts: 1,535
To Lennart:
Yes, CHARMM is a research tool for serious scientists that strive to know what they're doing, and making CHARMM "too easy" through things like "smart - in some situations - defaults" would encourage users to shoot themselves in the foot and blame CHARMM. Nevertheless, this principle doesn't justify the lamentable state of the documentation, nor CHARMM's general quirkiness. Just to name 2 examples: try to explain to a not-so-technical user why setting "FORMAT" under some conditions might break "CALC", or why "SYSTEM" calls with a lowercase variable in them only work when performed like this (mind the "back quotes"):
Code:
system "`echo shell_command @CHARMM_VARIABLE | awk '{print tolower($0)}'`"



That said, your point is taken; back to business.


To Sassy:
I have to agree with Rick. We had a cluster with a woefully obsolete version of the kernel and glibc in our lab, which we didn't wish to touch for "various reasons", so we:
1. compiled gmp, mpfr, and sufficiently recent versions of binutils and gcc from source in a user's home directory,
2. set appropriate environment variables to use the resulting utilities and libraries instead of the system's defaults,
3. compiled CHARMM,
4. copied the resulting CHARMM binary to a place where all nodes can see it (this works thanks to the fact that CHARMM is statically linked from A to Z).

Here's a "processed" version of my shell history when doing so. Please use common sense to point to the proper directories. So, after untarring all the tarballs from
http://gmplib.org/
http://www.mpfr.org/
http://www.gnu.org/software/binutils/
http://gcc.gnu.org/
in /home/user/install and cd-ing to /home/user/install, I did
Code:
######## Compile libraries and utilities ########
cd gmp-4.3.0
./configure --enable-mpfr
make
make check
cd ..
cd mpfr-2.4.1
./configure --with-gmp-build=/home/user/install/gmp-4.3.0
make
make check
cd ..
cd binutils-2.19.1
./configure
make
make DESTDIR=/home/user/gcc install
cd ..
mkdir gcc-4.3.3-bin
cd gcc-4.3.3-bin
../gcc-4.3.3/configure --enable-languages=all --with-gmp-dir=/home/user/install/gmp-4.3.0 --with-mpfr-dir=/home/user/install/mpfr-2.4.1
make bootstrap
mkdir /home/user/gcc
make DESTDIR=/home/user/gcc install
cd ..

######## SET ENVIRONMENT: ########
PATH=/home/user/gcc/usr/local/bin:/home/user/gcc/usr/local/libexec/gcc/x86_64-unknown-linux-gnu/4.3.3:$PATH
export LIBRARY_PATH=/home/user/gcc/usr/local/lib64:/home/user/gcc/usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.3.3
export CPATH=/home/user/gcc/usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.3.3/include



And now we're ready to compile CHARMM...

Note that going through the above procedure is not standard requirement for installing CHARMM. Things are a lot easier if the system is moderately up-to-date

Re: testfiles for c35b2 failed
Kenno #21198 06/10/09 10:09 PM
Joined: Nov 2005
Posts: 31
P
Forum Member
Offline
Forum Member
P
Joined: Nov 2005
Posts: 31
I'm currently looking for a fix for a problem and I came across this thread, so I'm chipping my two cents worth.

Regarding Lennart's point about user-friendliness my two biggest annoyances with charmm (and I know others share these) are that it's often very difficult to install on todays hardware (c35b2 seems to have solved my pains with opteron installs, but seriously, opterons have been around for ages!!) and the simple fact that the error messages are in most cases "less than useful" The amount of posts I and my collegues have sent to this forum to decipher a cryptic error message and have it turn out to be simple is mind-boggling!

Re: testfiles for c35b2 failed
PhilP #21199 06/10/09 11:01 PM
Joined: Sep 2003
Posts: 8,499
rmv Online Content
Forum Member
Online Content
Forum Member
Joined: Sep 2003
Posts: 8,499
I've found most of the 64-bit stuff (esp. for Opteron or EM64T chipsets) has been relatively stable since c33b1, although some of the commercial compilers were not well tested with that version.

The install.com procedure is somewhat arcane, compared to the "configure" and "make" steps for more standard distributed code.

The error messages are generally useful, but some are easily misinterpreted, esp. if one ignores earlier messages in the output.


Rick Venable
computational chemist

Page 1 of 2 1 2

Moderated by  lennart, rmv 

Link Copied to Clipboard
Powered by UBB.threads™ PHP Forum Software 7.7.4
(Release build 20200307)
Responsive Width:

PHP: 5.6.33-0+deb8u1 Page Time: 0.016s Queries: 42 (0.006s) Memory: 1.0345 MB (Peak: 1.1996 MB) Data Comp: Off Server Time: 2020-09-30 22:09:04 UTC
Valid HTML 5 and Valid CSS