Topic Options
#12510 - 11/22/06 02:35 PM extracting property time series from output logs
rmv Online   content

Forum Member

Registered: 09/17/03
Posts: 8314
Loc: 39 03 48 N, 77 06 54 W
I've attached a /bin/csh script to this post as a plain text file (getprop-csh.txt); to install it, first save it to a Unix/Linux machine, then

% mv getprop-csh.txt getprop.csh
% chmod u+x getprop.csh

where % indicates a shell prompt. I put it in my personal ~/bin subdir, but /usr/local/bin may also be appropriate.

The script will accept arguments, or prompt for them if not supplied; the first arg/prompt is for the property; the remaining args are treated as filenames. The filenames may also be entered at the second prompt. The property prompt is useful, as it lists the properties that can be extracted by the script. Files compressed with gzip are automatically decompressed by the script. A usage example:

% getprop.csh ENER dyn?.out.gz dyn??.out.gz dyn???.out.gz

The above will extract a potential energy time series from the indicated compressed log files. To do the same with the prompts--

% getprop.csh
TOTE TOTK ENER TEMP
GRMS HFCT HFCK EHFC VIRK
BOND ANGL UREY DIHE IMPR
VDW ELEC HBON ASP USER
HARM CDIH CIC RESD NOE
IMNB IMEL IMHB RXNF EXTE
EWKS EWSE EWEX EWQC EWUT
GEO MDIP SSBP SHEL DROFF
VIRE VIRI PRESSE PRESSI VOLU
XTLT SURF XTLP XTLM
A B C
ALPHA BETA GAMMA
PIXX PIYY PIZZ
PIXY PIXZ PIYZ
NORM
Property ==>
ENER
Be careful with wildcards, e.g. dyn?.out dyn??.out NOT dyn*.out
Filename(s):
dyn?.out.gz dyn??.out.gz dyn???.out.gz


For the above example the files ENER.DYNA.DAT and ENER.AVFL.DAT are created; the first has snapshot energies (every NPRINT steps) while the second has the averages and fluctutations (as a 3rd col), printed every IPRFRQ steps. Only the ENER.AVFL.DAT file can be used to compute correct averages with data from every integration step.

For extracting multiple properties for a large number of files, it is more efficient to do "batch" operations, as illustrated by the following /bin/csh script designed for use with a PBS queue:

#!/bin/csh
# RUN GETPROP ON FILES COPIED TO /scratch AND GUNZIPPED
#
#PBS -j oe
#PBS -N BatchProp

cd $PBS_O_WORKDIR

# copy to local disk and uncompress
set s = /scratch/$USER
if ( -d $s ) then
rm -rf $s/*
else
mkdir $s
endif
cp Out/dyn*.out.gz $s
( cd $s ; gunzip -f dyn*.out.gz )

# store current location
set pd = $cwd

# extract multiple properties
foreach p ( SURF VOLU ENER XTLT C )
~/bin/getprop.csh $p dyn?.out dyn??.out dyn???.out
end

# copy results back to original dir
cp *.DAT $pd

# cleanup
rm $S/*


Corrected "getprop" script posted below ...


Edited by rmv (04/09/11 04:22 PM)

Top
#12511 - 11/24/06 08:26 AM Re: extracting property time series from output logs [Re: rmv]
prakash Offline
Forum Member

Registered: 10/04/05
Posts: 206
I Have a file with all time dependent properties are written in md_run.mon file does this getprop script could be used for analysing it.

Top
#12512 - 11/24/06 12:20 PM Re: extracting property time series from output logs [Re: prakash]
rmv Online   content

Forum Member

Registered: 09/17/03
Posts: 8314
Loc: 39 03 48 N, 77 06 54 W
Without knowing how md_run.mon was obtained, it's hard to say, but probably not. The getprop.csh extracts data from the CHARMM dynamics output log files to a form that most plotting programs can handle. One can also use the KUNIT option to produce a huge file of all the energy terms, which may be what you have done. I prefer not to do that, as it adds to the disk storage requirement, and the data is already included in the output log file. Also, I believe the KUNIT data is only the instantaneous snapshot values from every NPRINT steps. To get the true average of a simulation property, the data printed every IPRFRQ steps in the output log are needed; these values are averages that include every integration step, and are not otherwise available.
_________________________
Rick Venable
computational chemist


Top
#12513 - 11/24/06 01:26 PM Re: extracting property time series from output logs [Re: rmv]
prakash Offline
Forum Member

Registered: 10/04/05
Posts: 206
thank you

My md_run.mon file is a snapshot of energy terms stored on kunit integer so i got it that csh script doesn't work.

I have the md_cpt_run.out as an output file further how could i use the csh script to have energy plot frm the output log file.
I tried something like

./getprop.csh md_run_cpt.out
'nknown option: `-
Usage: csh [ -bcdefilmnqstvVxX ] [ argument ... ].


how could i run this csh for my dyn*.out

thanks

Top
#12514 - 11/24/06 02:04 PM Re: extracting property time series from output logs [Re: prakash]
rmv Online   content

Forum Member

Registered: 09/17/03
Posts: 8314
Loc: 39 03 48 N, 77 06 54 W
There are at least two problems--

[1] The first arg must be the property to be extracted, and cannot be omitted if a file name (or names) is also specified.

[2] The error message is from /bin/csh and indicates that the first line of the script may have become mangled somehow; it should be

#!/bin/csh -f

but the error message suggests it is not.

Depending on how the "getprop-csh.txt" file was downloaded and transferred to a Unix or Linux machine, there may be line termination issues which might cause this error as well.

Try running the script w/o any args.
_________________________
Rick Venable
computational chemist


Top
#12515 - 09/08/07 07:43 AM Re: extracting property time series from output logs [Re: rmv]
jeffrey Offline
Forum Member

Registered: 03/18/07
Posts: 148
Hi Rick,

There may be a bug in the script when the total energy is so large that no space exsits between $3 and $4. An example is as following:

DYNA DYN: Step Time TOTEner TOTKe ENERgy TEMPerature
DYNA PROP: GRMS HFCTote HFCKe EHFCor VIRKe
DYNA INTERN: BONDs ANGLes UREY-b DIHEdrals IMPRopers
DYNA EXTERN: VDWaals ELEC HBONds ASP USER
DYNA IMAGES: IMNBvdw IMELec IMHBnd RXNField EXTElec
DYNA EWALD: EWKSum EWSElf EWEXcl EWQCor EWUTil
DYNA PRESS: VIRE VIRI PRESSE PRESSI VOLUme
DYNA XTLE: XTLTe SURFtension XTLPe XTLtemp
---------- --------- --------- --------- --------- ---------
DYNA> 75 10.14000-162151.67396 60025.16076-222176.83472 292.64309
DYNA PROP> 10.82789-161557.96764 61806.27973 593.70632-130235.34232
DYNA INTERN> 5209.02536 7489.89266 626.63849 2788.69620 314.08991
DYNA EXTERN> 24072.57374-179978.33286 0.00000 0.00000 0.00000
DYNA IMAGES> 868.70118 -10728.85476 0.00000 0.00000 0.00000
DYNA EWALD> 6505.1645-1837456.4041 1758111.9749 0.0000 0.0000
DYNA PRESS> -8858.88544 95682.44699 908.53397 14038.57832 668591.80549
DYNA XTLE> -168997.11752 -96.34063 9.75465 6211.60003
---------- --------- --------- --------- --------- ---------

It seems that the command (grep "$ty> " $t | awk '{print $3, $4}' > p.t) will regard $3 and $4 as $3. So if I try the command: getprop TOTE t.out

the result is: 10.14000-162151.67396 60025.16076-222176.83472, which is not required.

Is there anyway to treat above situation?

Thanks very much.

---------------

Jeffrey

Top
#12516 - 09/08/07 11:43 AM Re: extracting property time series from output logs [Re: jeffrey]
rmv Online   content

Forum Member

Registered: 09/17/03
Posts: 8314
Loc: 39 03 48 N, 77 06 54 W
The script is supposed to change '-' to ' -' to avoid this problem, at least for large negative numbers. I'll have to check that ...

It looks like my initial post was an earlier version, w/o that correction; the corrected version is attached.



Attachments
15481-getprop.txt (1310 downloads)


Top
#12517 - 09/09/07 02:20 AM Re: extracting property time series from output logs [Re: rmv]
jeffrey Offline
Forum Member

Registered: 03/18/07
Posts: 148
The corrected version works well now. Thanks very much, Rick.

Top
#24462 - 05/31/10 04:35 PM Re: extracting property time series from output logs [Re: jeffrey]
JohnPenniston Offline
Forum Member

Registered: 01/27/06
Posts: 69
Loc: Boston, MA
I've downloaded it and used it. It works very well and saved me a lot of trouble. Thanks!:-)
_________________________
John Penniston

Top

Moderator:  chmgr, John Legato, petrella