Wednesday, September 30, 2015

command "cat": clean and powerful

There are several commands that are very powerful; such as the command "cat".

The word cat comes from the word concatenate; it is commonly used in various languages.  For example, cat in unix connects/links the file content to the monitor. 

 

The cat command can be "infinitely" long; it contains/links a mixture of expressions and statements; the last one is normally an expression, which happens to be the return value.  (sometimes, I wonderwhat  if the right most one is a statement; does it evaluate and return? e.g a=7  does it return a true or false or return a value for the assignment )

Here is an example:

cat(s==0,i==1,while(le(i,N),s==s+p[i],i==i+1), 5*s)

the command line contains two assignments, one loop, one return value.  In plain English, the above line says that we assign 0 to s and 1 to i; then we check whether i is smaller or equal to N or not, if yes, we add up all the p elements, and move up the counter i to the next one; when this thing done, we evaluate the expression 5 multiplied by s and return it if there is a holder for it.

if we write in C++, it may look like this

for (s=0, i=1; i <=N; i++) s+=p[i]; return 5*s;

For a simple loop, one may wonder why bother using cat (my kid used to ask me why write several lines to calculate the sum of 3 numbers, in a similar way); the beauty of using cat is for complicated cases; it is very powerful and it is cleanly structured.  moreover, nesting is permitted.

Imagine the whole thing is named; how more powerful can we get? 

I personally admire the beautiful mind from where these nice things flew out. 


Monday, September 28, 2015

The strengths of TOPAS

Dr. Wang, an expert in Topas, of Bruker, would like to emphasize that Topas has many advance features.

Listed here are some of them:

  • Topas has a batch command on top of the GUI and kernel; and Topas BBQ is widely used in cement industry and mineral industry
    • peak file parameters are not refined, i.e. the peak profiles are based on the physics, such as the the Xray tube, radii, all slits size.  "The fundamental parameter approach in TOPAS also use convolution based peak shape and calculate instrumental peak broadening based on physical parameters like divergence slit size and goniometer radius etc. The actual fundamental parameter approach is published by the author of TOPAS: Dr. Coelho himself."  
    • background refinement is automated. 
    • There is no hard limit on the number of phases, parameters, and data points for refinement
  • Both spherical harmonics and MD are implemented to handle preferred orientation
  • Line broadening are modeled.
  • Strongly disordered kaolinite and turbostratically disordered phases can be handled. Available in various versions.
  • Mircoabsorption is corrected wit Brindley model since v1.
  • "TOPAS has implemented both Internal standard method and External standard method, and PONKCS method for quantification of phases without structure model"
  • Many other special features, such as defining rigid and non-rigid bodies, the rotation, translation and torsion of the whole or parts of them, bond length linked to atom occupancy.

On top of these advance features, convenience features are also there.  For example, all the keywords are written in English and self explanatory; and  running the refinement needs only two files ( the raw data, .raw file and the input, .inp file).

Because of these and many other possibilities TOPAS offers, the usage of TOPAS is much more wider.  A search in the International Union of Crystallography themselves would show the current the number of citations.  The following is an example, showing a high number of appearance.  http://journals.iucr.org/ 








Sunday, September 27, 2015

TOPAS and BGMN: batch files

If you have a measurement file, e.g diffractada_sample123.raw (a bruker raw file, the name can be anything; I normally put a sample identification phrase, such as where it is from and what depth it was at), you would like to refine it with different parameters or different combinations of minerals.  You can write a batch file and let the program run thru the weekend.  or you can run many samples this way, too.

For BGMN, running from GUI or from command line does not make a difference.  For Topas, running from command line has much more flexibility than running from GUI; this was foreseeable likely due to that the command line and the GUI are from different owners (?)

Because I write the blog posts at home and TOPAS is available only at workplace; I cannot put screenshots here to show how it is done for TOPAS.  Sorry.

________________a.bat___________

rem  make sure the path is set up correctly
rem make sure the output file is NOT overwritten
rem  __________________________________
BGMN   diffractada_sample123_paramset1.sav
BGMN   diffractada_sample123_paramset2.sav
BGMN   diffractada_sample123_paramset3.sav
TC   diffractada_sample123_paramset1.inp
TC   diffractada_sample123_paramset2.inp
TC   diffractada_sample123_paramset3.inp


for example:
I can run the example file with two mineral assemblages (see figure; the combination does not make sense; just to demonstrate what the program can do)

by putting the following line in a bat file

rem  ____copy from here and paste into a notepad_
rem  __________to generate_a.bat______________
rem  ____you have to make sure that all three mineral structure files
rem _____are present in the folder
BGMN  BariteQuartz.sav
BGMN  BariteCalcite.sav
rem ______copy to this line________________



Friday, September 25, 2015

BGMN: what files are there

(all features are described in BGMN.de; I am rehashing what the experts are saying over there;
the flexibility is apparent as the language is structured and free-styled; but, the variables are case-sensitive)

One of the main nice features, in my opinion, is the calculated instrument profiles at various 2theta values.  These profiles are further interpolated.  To generate these profile files, an instrument .sav file (which you can open with notepad) is needed. you'll see that the instrument setup file, .sav, and two profile files are needed for the refinement.  Certainly, if you don't have any profile, you still can do the refinement, and the error can come out of this.

So what files are needed to do the refinement?

Here is a look at one of the exercise folders.  There are two rows of files; the top row has two .str file, one .sav file, and one .val file; the bottom row has three files with the same name, and they are for the instrument profiles.




The .val file is the measurement file from your diffractometer.  It is simply a file telling you the diffracted x-ray intensity.  The two .str files are the mineral structures, in our example, the structures of quartz and barite.  The top row .sav file, BariteQuartz.sav, is a control file, containing instructions you are telling the program how to run the refinement.

My guess is that Dr. Kleeberg wrote these files; all of them are documented super well.  Let me open the  BariteQuartz.sav file.



the measurement data file looks like this
 
the structure file, quartz.str, is here;
 
 

Thursday, September 24, 2015

BGMN: the GUIs and unique features

There are two GUIs:  Dr. Doebelin's Profex and Dr. Bergmann's BGMNwin.

 Dr. Bergmann's BGMNwin was easy to use.  It is the file BGMNwin.jar.  For most window users, you just need to double click on it.  If it does not work, open the black command window, go to the BGMNwin directory, using cd command; then  type java -jar BGMNwin.jar;  looks to me that it was written in java, so you need to make sure you have java installed on the computer.

http://profex.doebelin.org/ is the web address of Dr Doebelin's profex, which is a front end of BGMN (fullprof).  it is very user friendly.  you don't need to be computer savvy to go through the procedure.  there are class notes and handouts to help understand the concept. 

______________Unique features________________

http://bgmn.de/features.html says


BGMN has an impressive set of features which are not to be found in this form in other Rietveld programs. 

  • Fully automatic calculation which requires no user interaction
    • best numerical algorithms specially designed for BGMN which guarantee an extreme numerical stability
    • physical based peak model which eliminates the refinement of peak profile parameters (no more refinement of u, v, w)
    • automated background refinement
    • unlimited number of phases, parameters, measuring points etc.

  • Correction of strong and multiple preferred orientation with spherical harmonics:
    • can correct the strong preferred orientation of phyllosilicates
    • can correct the PO of phases with multiple cleavage
    • automatic decision about applicability of complicated models

  • Pre-defined models for microstructure related line broadening effects:
    • refinement of crystallite size broadening effects with 1-2 parameters per phase; 6-7 in anisotropic case
    • refinement of microstrain with one parameter per phase; 6 or 15 in anisotropic case

  • Phase analysis of disordered layer silicates/clays by implementation of advanced real structure models (e.g. for kaolinite and chlorites)
    • succesful description of strong disordered kaolinite
    • modelling of turbostratically disordered phases possible, e.g. smectites
    • recursive modelling of random stacked clay minerals etc.

  • BGMN now is multithread-enabled and therefore gains in speed from multicore PCs/workstations.
    • In special, this is the solution for bringing time-consuming recursive structure models into the reach of daily laboratory analysis.

  • Correction of microabsorption with the Brindley model
    • only the mean particle size has to be provided

  • Calculation of amorphous content by an internal standard method possible
  • Structure interpreter language for free definition of new parameters and constraints between parameters
  • This language includes special features for structure investigation and refinement:
    • enables the definition of rigid and non-rigid bodies
    • non-rigid bodies may be distorted by shifting, rotation or torsion of the whole or parts of them
    • built in force field model to ensure optimal distances between the atoms and to guarantee chain concatenation
_____________

Papers by any one of these are joy to read, from my personal experience.
J. Bergmann; R. Kleeberg; K. Ufer; P. Friedel

Wednesday, September 23, 2015

BGMN: 1st time use for dummies

Nowadays, there are GUIs so you can click and everything is there.
In late 1999, I was trying to learn Koalariet and BGMN.  and started to like BGMN (bgmn.de) because of the simplicity and nicety.  I still remember the 1st time of try. 

1. click on start, type "cmd" to bring a dos window; type drive letter and  then enter (the drive you save the data on, in my case it is Q:  ).  if your files are already on C: drive, you don't have to type this command, D: E: or Q: .
 


 
2. In the black window, type set path=%path%; (note: there is no trailing space) then drag the BGMNwin folder in the right explorer window into the black box; then press enter.
 
 
 
3. in the black window, type set eflech=  (note there is no trailing space) then drag the BGMNwin folder into the black window, hit enter
 
4. in the black window, type cd [space]  , then drag the folder where the exercise folder is, hit enter
 
 
 
5. in the black window, type BGMN [space] then drag the ".sav" file, hit enter
 
 congratulation to you! you did it
 

TOPAS from Bruker and Topas Academic


Bruker provides the commercial version and Dr. Coelho provides the academic version.  For non-clay minerals, it is very straight forward using its GUI; but all the advanced usage is to be done in the non-GUI mode.  A good editor is needed so all the reserved keywords can be highlighted for efficiently edit the file.  The following website recommends JEDIT, which is free. 

[Out of curiosity, I searched Koalariet, it is still there in the web space. It was also written by Dr. Coelho, the structure and input files are the same.  If you want to get a taste of Topas but you don't have money, you may want to give this a try.  It is free, or was free. I'll give it a try soon in the future for the economic users.
   http://ccp14.chem.ucl.ac.uk/ccp/ccp14/ccp14-by-program/xfit-koalariet/xfit-koalariet-win95/read1st.txt]

http://www.topas-academic.net/
 
The latest version is version 5.  what is new in version5.pdf
 

Monday, September 21, 2015

Rietveld Software: GSAS

according to Dr. Toby, there are dozens of software.

But of the people they surveyed, 3/4 are using GSAS.  Certainly, it is developed by a group of smart people and it belongs to our government, i.e. you can use it free of charge.  so "i don't have money" is not an excuse anymore.

here is the full blown intro by Dr, Toby

http://www.aps.anl.gov/Xray_Science_Division/Powder_Diffraction_Crystallography/4GSASEXPGUIintro/4GSASEXPGUIintro.html
 
there are online trainings too. you can go to
http://www.aps.anl.gov/Xray_Science_Division/Powder_Diffraction_Crystallography/GSASEXPGUI.html

_______________________________________
one Chinese guy is giving some advice, in  a funny way.  Hope you heard of google translate; I 'll translate some of the sentences when feel like doing it.
http://emuch.net/html/201110/3668889.html

关于精修,我想说的是,一定要知道X射线晶体学的一些理论知识,如果对XRD的认知仅仅停留在会对照PDF卡片定性分析,建议暂时不要入手精修,先闭关修炼一段时间再来入手。 
我的经验是这样的:一年前,我对XRD的认知就是会定性分析,用下jade,highscore什么的。偶然的机会下了GSAS,弄了两下,一头雾水。 后来,知道了倒易空间,布拉菲点阵啊,32点群,230对称群,11laue群啊什么的,(虽然不精通,但至少知道怎么回事)。 后来再用了下GASA,半天搞定,后面又用了一两天 搞定了 fullprof,muad,topas(都是触类旁通)。 现在完全可以用这些软件进行精修。

这些软件原理基本一样,但各有特色:

GSAS和fullprof名气应该很大,个人觉得GSAS的择优取向校正 多了 sphere harmonic函数,这个函数校正择优取向很有效,fullprof虽然提供了4中择优校正函数,但是都要自己输入择优方向,而harmonic只要选择阶数和laue群就行,对一些取向不是特别明显 或者通过谱图,无法肉眼判断择优方向时,很有效。【JZ: both GSAS and fullprof are famous; but personally I think GSAS used spherical harmonic function for orientation correction, which is very effective.  On the other hand, fullprof provides four different orientation corrections, but you have to select which direction is the preferred orientation.  Spherical harmonic correction needs only two inputs, the order number and laue, and it's useful for samples hard to judge which direction is the preferred direction 】而GSAS的微结构分析方面个人觉得比fullprof差(也或许是我不会,呵呵),fullprof 提供 了 各项同性和各向异性的 晶粒大小和 微观应变的 模型,只要提供 irf文件,就可以进行 WH和WA 的分析,非常方便。(但是对微结构的分析 我还推荐一个专门软件:MarqX和PM2K,这两款款是同一个组编写的,PM2K界面要友好,都是免费软件,只是需要自己输命令,比较麻烦,但做微结构分析,这绝对是很好的软件)。
再来说说maud,可能有些人不熟,但却是很好用。这里面提供一个向导,就是说你只要点一个键,就可以完成精修,很傻瓜式,但是要得到好的结果,只用向导肯定是不行的。但是不用向导就比较麻烦,因为各个参数全部在一个叫parameter list的树形目录里,不是按钮模式,是文字模式,可能 对不懂理论的人 就傻眼了。 不知道写的是什么。但是内行,应该很看出 大部分参数的意义。这个软件 对择优取向的模型非常多,harmonic里 提供了 11个lau群( 而GASA里 好像只有 3个?反正很少),而且能画出彩色极密度图,非常方便。对微结构分析至少比GSAS方便。只是峰形函数只有一个,没有GSAS和fullprof多。

那个最后一个就是topas了 我用的是 academic版, 所以运行起来麻烦, 总体感觉跟GSAS差不多,择优取向 也有harmonic函数,但是 好像 背景不能自己加点。 但是有个问题:可能是这个收费软件 即便是academic版,我用的是破解版,输入 AlOOH模型时,计算的向相对强度 始终不对,我检查了占有率和温度因子,好像都没错误,不知道为什么?  是不是破解版缺少了 什么东西,或者是原子的散射因子 数据库有问题?  反正不推荐使用,因为在精修方面 ,它能做的 上面三款 都能做, 而且 它还是收费软件,强烈不推荐。

Students in Clay minerals and XRD


I did not know there was this society for students.  Watch this video

https://www.youtube.com/watch?v=IxftaP9I-sg#t=31
<iframe width="560" height="315" src="https://www.youtube.com/embed/IxftaP9I-sg" frameborder="0" allowfullscreen></iframe>


_________________________________________
http://www.master-imacs.org/
http://www.master-imacs.org/master-nanotechnologie-overview-Obj/obj.php
IMACS (International Master in Advanced Clay Science) is the first multi-disciplinary program that brings together the widely-distributed knowledge of clay science.
IMACS is an integrated 2 years master programme delivered by five Universities : the University of Poitiers (France), the Technical University of Crete (Chania - Greece), the University of Aveiro (Portugal), the University of Ottawa (Canada) and the Federal University of Rio Grande Do Sul (Porto Alegre – Brazil), and is supported by the French Clay Group (GFA) and AIPEA (International Association for the Study of Clay Minerals) which are associated members of IMACS.

_________________________________________
some young guys sharing their experiences.

Sunday, September 20, 2015

Coming back to XRD business, I feel grateful to J Bergmann


In the past decade, I wrote many critiques and proses on daily stuff.  But last week, my son said I could be the another Khan after I posted a video about Khan's Academy.  That was quite encouraging to me.  Maybe I should jot down what I do to make a living and hope this may help one or two young guys.

______________________________________________________

Back to x-ray diffraction (XRD) from log interpretation, I have to twist the mind again on atomic position, structural factor, etc., again. In the past, I tried different Rietveld software, and I liked and still like BGMN the most.  Go to BGMN.de to find the nice things about the software.

Ufer et al 2012 talk about the 1D oriented simulation, and he summarizes the nice features of BGMN.

"BGMN contains an interpreter language which allows the definition of additional functions and parameters . The manipulation of the complex structure factors o f multiple layer structures within a single unit cell, the so-called sub-phases, is also possible. The use of an interpreter language allows a very flexible and transparent formulation without the necessity of hard coding. A super-cell approach was used to calculate the structure factors of these sub-phases periodically in the a and b directions and aperiodically in the c direction. "

_____________________________________________________
Unfortunately the author of the software, a genius in my opnion, passed away a few years back at an early age.  here is what his colleague said about him (http://www.iucr.org/resources/commissions/powder-diffraction/?a=41899)

Joerg Bergmann (1956-2010)_____________
"Joerg Bergmann studied physics in Dresden and in 1984 he presented his Ph.D thesis with the translated title "Contributions to evaluation and experimental design on X-ray powder diffraction" (http://www.bgmn.de/download/thesis.pdf).

Joerg was an exceptionally gifted mathematician.  He developed a solution for convolution-based peak profile modeling, which was integrated in a peak search program and combined with options for optimized measuring strategies.  Unfortunately, these results could not be published before the German reunification, at the EPDIC 1 in Munich (Querner et al., 1991).

Motivated by colleagues and friends, Joerg started development of a Rietveld program based on his "fundamental parameter" peak shape model in the early 90s.  I was as an inexperienced user of DBWS and was shocked when I first tested Joerg's software in 1993.  Some mysterious things happened, and the expected results were all written in the results file.  I suspected there were some dirty tricks or that some hidden parameters were fixed, but the great convergence behavior could be consistently reproduced.  This high level performance remains as one of the fascinating characteristics of BGMN to date.

Despite his poor health, Joerg worked tirelessly on the BGMN program as a private project and hobby without significant institutional support.  There are numerous impressive results and publications obtained with Joerg's software that have been noticed by the community including the IUCr indexing round robin (http://sdpd.univ-lemans.fr/sdpdrr2/sdpdrr2-CPD.pdf) and the CMS Reynolds Cup in quantitative mineral analysis (http://www.clays.org/SOCIETY%20AWARDS/RCresults.html).

Joerg's BGMN website (http://www.bgmn.de/methods.html#cite) also contributed to his reputation in the community of crystallographers and X-ray powder diffractionists and unselfishly provided valuable and practical information to people working in the field.

Joerg was known as a great personality, open in discussions and helpful in solving any scientific problems.  Sometimes it was hard to follow his highly technical thoughts, but he patiently answered any questions.  He was impressively fast at fixing bugs and implementing new features of his software that were related to someones individual scientific problem. 

Outside the scientific world he loved singing in the University chorus, hiking, and cooking exclusive meals. Joerg was also a passionate photographer and artist.  Some of his photos and computer graphics can be seen here http://www.jbergmann.de/.  Joerg's awareness about his unstable health allowed him to live intensely with the time he had and share the beauty and passion in his life with his beloved wife Anne.

Joerg Bergmann will be sorely missed by friends and colleagues in Germany and around the world.

Reinhard Kleeberg, Freiberg, Germany (October 2010)"

_____________________________________
I feel so privileged to talk about his software.  My last communication with the him was in 2009, a few month after I went into log analysis.  (I don't have a Dr. in the title). 

On Wednesday, July 1, 2009 12:15 AM, Joerg Bergmann <email@jbergmann.de> wrote:
Dr. Zhang:

I have added right now that function to BGMN.
It was for optimization purposes in conjunction
with the new function F(phi...) as a core of an
numerical integration (function cintegral).
The name of the new function is T2, the same has been
doen to D and the new function is called D2.

Greetings from Dresden

Joerg

Am 03.05.2009 19:56, schrieb ji zhang:
> Dr. Bergmann:
>
>
> I have another suggestion;
> can you add more positions at the T function such as
> T(6 parameters of rotation and translation; old positions; new positions)