SmartProof5 - Control Microscope for Surface Roughness (multiple locations across XY)

Discuss macros to control the hardware functions of your ZEISS microscope
user-7045
Posts: 7
Joined: Thu Jan 01, 1970 1:00 am

SmartProof5 - Control Microscope for Surface Roughness (multiple locations across XY)

Post by user-7045 »

Dear all,

I am fairly new with the OAD and just got the software update that now unable to run macro amongst other new features.
I would normally have tried to run it in the workflow, but it is not possible to split and SAVE multiple tile measurements as SURF (splitting work, but not the saving part). In addition, it was not communicated how to import tile location. Therefore I ended up writing a macro to do most of the heavy lifting (i.e. get the information required - import the data required (pb with this) - move to location - run the experiment (pb with that) - and save the data after an acquisition is finished (pb related to above)

I would also like to import the tile location (XY). I normally would have use pandas package in Python, but it does not work (even after amending the sys path). I also need to manipulate the table afterwards so pandas is required here.

I have found out how to move the table XY - control the focus, but it does not seemed possible to move the Z-axis (which is ok in this case, but would be good to know how). So I can go to the location I want to acquire. I can ran autofocus and acquire the live image.

Then I would like to run, if possible, the acquisition (surface roughness) from the macro and save the data as SURF using the filename from the CSV for post-analysis using MountainsMap. I read that it is possible to run an experiment from the macro - but I don't have the example related to smartproof or pdf.

If easier, I can provide the code, I have been working on to exemplify my struggles.
I have also be trying to find examples (two scripts were provided with little use) and no pdf with the fonctions details

Would appreciate some help
Best
--
Olivier
user-4
Posts: 398
Joined: Thu Jan 01, 1970 1:00 am

Post by user-4 »

Hello Olivier,

could you please send us your macro because it is not so easy for us to estimate how much of your programming is alredy done and what is still needed.

Thanks!
user-7045
Posts: 7
Joined: Thu Jan 01, 1970 1:00 am

Post by user-7045 »

Here the script I wrote - I commented out some sections from earlier trials.
Please note at the end of the script part that I need help with, ie.
- define a tile 2*2 based on current locations with the center at x,y
- define all required parameters to launch the acquisition (similar to 3D-Tiles (measurement area) in smartproof)

Code: Select all


#######################################################
## Run 18 tiles
##
## Macro name: Tiles run- run each tiles available and three acquisition per tiles at different locations
##
## Required steps:
## need to have a csv file which include all the name of the tiles to be analysed
## need to have a csv file with all the position for the 18 tiles
##
## Required modules: no
##
## Ask the user to provide project number and SAP data.
##
#######################################################


Zen.Application.Documents.RemoveAll()

########
### This does not work - how to import other packages
########

import sys
sys.path = sys.path[0:5]
#sys.path.append(r'C:\Program Files\Carl Zeiss\ZEN core\IronPython\Lib\site-packages')
sys.path.append(r'C:\Program Files\Carl Zeiss\ZeissPython\Py20190211\env\Lib\site-packages')
import datetime
import csv
#import pandas as pd
#import numpy as np

## activate IO library
from System.IO import File, Directory, FileInfo

## create setup dialog to enter information about project 
window = ZenWindow()
window.AddTextBox('tval0','Project Number               ','000')
window.AddTextBox('tval1','Prod Order                   ','a000')
result=window.Show()


## check, if Cancel button was clicked
if result.HasCanceled == True:
    sys.exit('Macro aborted with Cancel!')


## create setup dialog to export data to the folder with the SAP number
filename = result.GetValue('tval1')
outpath = 'D:\\Fileserver'+'\\'+str(filename)


## load the CSV file with all the locations
#locdata = pd.read_csv('D:\Users\zeiss\Desktop\tileslocXY.csv')
#data.head
f = 'D:\\Users\\zeiss\\Desktop\\tileslocXY.csv'
tab = ZenTable()
tab.Load(f)

## create a setup dialog to enter the tiles that will get analyse
## number will be used to defined the naming convention

window = ZenWindow()
window.AddTextBox('tval2','Start Tile               ','001')
window.AddTextBox('tval3','Finish Tile              ','018')
window.AddTextBox('tval5','Surface State            ','000')
window.AddDropDown('tval4','Select the side         ',['s1','s2'],0)
result=window.Show()

start = int(result.GetValue('tval2'))
finish = int(result.GetValue('tval3'))
state = result.GetValue('tval5')
side = result.GetValue('tval4')
side = '_'+side+'_'

if start<=0 | finish <=0
    sys.exit('Macro aborted: Start or finish cannot be negative')
elif start>finish:
    iter = start-finish
elif start<finish:
    iter = finish-start
else:
    sys.exit('Macro aborted: Start and finish cannot be identical')
    # technically possible if only one tile is place - need to be considered later.

##
# append the tileslocXY table
arnum = list(range(start,finish+1,1))
arnum = arnum+arnum+arnum
arnum = sorted(arnum)
    # need to append arnum to the tileslocXY in the 5th column
    #locdata = pd.concat([locdata,pd.DataFrame('Tile Nbr': [arnum]),axis=1])


now = datetime.datetime.now()
date = now.strftime('%Y%m%d')
fn = date+'_'+filename+'_' ### e.g. 20192709_1520570_
fnlist = [fn]*58
stlist = [state]*58
slist = [side]*58
llist = list([10,25,40])*18 ## 18 is hardcoded - should be changed later
rlist = ['_Rough']*58

#locdata = pd.concat([locdata,pd.DataFrame('Fn list': [fnlist]),axis=1])
#locdata = pd.concat([locdata,pd.DataFrame('S list': [slist]),axis=1])
#locdata = pd.concat([locdata,pd.DataFrame('L list': [llist]),axis=1])
#locdata = pd.concat([locdata,pd.DataFrame('R list': [rlist]),axis=1])

tab.Columns.Add('Fn',str)
tab.Columns.Add('Tile Nbr',str)
tab.Columns.Add('St list',str)
tab.Columns.Add('S list',str)
tab.Columns.Add('L list',str)
tab.Columns.Add('R list',str)
tab.Columns.Add('Name list',str)
#locdata['name list'] = locdata[['Fn list','Tile Nbr','S list','L list','R list']].apply(lambda x: ''.join(x),axis=1) ### e.g. 20192709_1520570_001_s1_10_Rough
#tab.Rows.Add(fnlist+arnum+stlist+slist+llist+rlist)

Zen.Application.Documents.Add(tab)

#rc = tab.RowCount
rc = len(list(range(start,finish+1,1)))*3
HS = Zen.Devices.Stage
FAP = Zen.Devices.Focus.ActualPosition

HS.MoveTo(0,0)

for ii in range(1,rc,1):
    ## Drive the stage to specific location
    # x and y will come from the csv file entitle 'D:\Users\zeiss\Desktop\tileslocXY.csv'
    x = float(tab.GetValue(ii,1))
    y = float(tab.GetValue(ii,2))
    
    tab.SetValue(ii,3,fnlist[ii])
    tab.SetValue(ii,4,arnum[ii])
    tab.SetValue(ii,5,stlist[ii])
    tab.SetValue(ii,6,slist[ii])
    tab.SetValue(ii,7,llist[ii])
    tab.SetValue(ii,8,str(fnlist[ii])+str(arnum[ii])+'_'+stlist[ii]+slist[ii]+str(llist[ii])+str(rlist[ii]))
    
    # to change the objective
    OC = Zen.Devices.ObjectiveChanger.ActualPosition 
    #
    HS.MoveTo(x,y)
    # should have check to make sure it is arrived 
    
    
    ########
    ### How can I do this either here or as part of the macro
    ########
    ### define a tile 2*2 based on current location with center at (x,y)
    ### define some parameters to launch the 3D-Tiles (measurement area) smartproof acquisition
    ### or launch application from here
    
    ### save the data according to filename as SURF

    
    # set active for debug
    Zen.Application.ActiveDocument = image

    newPathAndFile = outpath + "\\" +  tab.GetValue(ii,8)
    image.Save(newPathAndFile)

## end of loop

user-4
Posts: 398
Joined: Thu Jan 01, 1970 1:00 am

Post by user-4 »

Hello Olivier,

thanks for the code! Now it is more transparent what you are doing.

First of all pandas seems to be for ‘normal’ Python (unmanaged code), not IronPython (managed code / .Net). As Zen is written in managed code we have to use IronPython in the macro editor and cannot include packages from normal Python. A workaround is perhaps to save the data in a file and start via Shell Command (in IronPython) the Analysis with Pandas in ‘normal’ Python etc.

As far as I understand your code I would recommend to have a close look at the Acquisition Tab and configure a Tiles-Experiment with tiles manually added as Positions. See also the options available.

After playing around with tiles experiments you can begin to work with the experiment in a macro.
Maybe the most important lines of code to get a quick start are

Code: Select all

zenExperiment = Zen.Acquisition.Experiments.CurrentItem
zenExperiment.GetSinglePositionInfos(0)
zenExperiment.AddSinglePosition(…)
Your approach with ‘Drive the stage to specific location’ is very basic and there are certainly some situations where this is really needed but on the other hand you might ‘reinvent the wheel’ if the general approach via tiles-experiment also works for you.
user-7045
Posts: 7
Joined: Thu Jan 01, 1970 1:00 am

Post by user-7045 »

Hello,

I will try the starter code you provided, but I foresee me coming back to you for more details.
Can you give me a .czjob or .czexp file which works so that I can see the differences between the one I have exported (see below)

As per you comment, I did try to use the workflow at first for this, but I run into a few problems:

1. I have 18 specimens (which are each measured at three different locations) and for each, I have a 2*2 tiles (~2x2mm using 10x obj) on the XY table and the scenes stopped (software crash) after 7 - memory overall - we will increase the RAM on the computer - but it will not work for 58 measurements
2. Unable to perform an autofocus and autocontrast for each tileset and centered on Z - which means it is still highly hand-heavy (I have multiple set of 58 tiles measurements to carry)
3. Unable to save each tile (after splitting per scenes) the surface automatically with a changing filename

Hence this is the reason why I choose to go through the OAD way which should enable me to do all of the tasks above
my questions are as follow :

Can you provide me with an example as to how to carry a 3D roughness acquisition or a functioning *.czexp or *.czjob

1. Perform Autofocus on the location
This works :)

Code: Select all

from System import ApplicationException
from System import TimeoutException

try:
    z = Zen.Acquisition.FindAutoFocus()
except ApplicattionException as e:
    print(e.Message)
except TimeoutException as e:
    print(e.Message)
2. Perform Autocontrast at the defined focus height
This DOES NOT work :( - value is apparently too high - which caused an error
EDIT: This was resolved, i.e. no error when running with an ZenExp inside the bracket

Code: Select all

from System import ApplicationException
from System import TimeoutException

try:
   z = Zen.Acquisition.FindAutoExposure()
except ApplicattionException as e:
   print(e.Message)
except TimeoutException as e:
   print(e.Message)
3. Define a 2*2 tile set - centred on XY - I am using the x10 objective
DOES NOT work :(
Important note, I suppose:
In the case of running ZenCore SmartProof - need to use the .czjob rather than .czexp
As far as I am aware they are the same when viewed in notepad.
I also tried to run an easy 2D image acquisition experiment - got the same error - see below

When I ran the following code - which I assumed should ran smoothly ...

Code: Select all

ZenExp = Zen.Acquisition.Experiments.GetName('pathfile.czjob')
ZenExp.SetActive()
image  = Zen.Acquistion.Execute(ZenExp)
I get the error (this exp works in the normal workflow)-
Experiment to execute is not valid. Please check your settings. The following invalidities were found:
No Channels are defined or activated

Please advise

4. Run the acquisition for a specific Zrange - centred on the Zfocus - I am using the fast acquisition to reduce the amount of layer

5. Save the data using the filename as SURF for further analysis in MontainMap

Code: Select all

Zen.Application.ActiveDocument = image
NewPathFile = outpath + '\\' + tab.GetValue(ii,8) + '.SUR'
image.Save(NewPathFile)
Thank you
user-7045
Posts: 7
Joined: Thu Jan 01, 1970 1:00 am

Post by user-7045 »

Any news ideas for these problems.

I realised that some of the solutions provided are unique to light microscope, but does not apply to SmartProof - for example Scenes Splitter.
I am still having problem.

Thank you
Regards
user-7045
Posts: 7
Joined: Thu Jan 01, 1970 1:00 am

Post by user-7045 »

As suggested by CZM3 above, I created from a template with all of the locations (54) for all the different specimens I have on the X&Y tables which was then followed by a OAD script to save each of the scenes in the files.

This never worked as the program failed to execute every single time and stop from Acquisition Error.
This means that I lose all the work that was acquired before --> One of the reason why I want to ran everything in OAD

Acquisition Error :Acquire CompositeZStack Imagesand FocusPositionsDiffersCompositeZstack .
Please note that the program worked each time when I work with three differents locations.

So I decided to set less tiles and separate them in three acquisition within the same workflow as I thought that it may be linked to the fact that the image is too large. This run ok-ish and is very slow - moslty because of the fact that I cannot do an autofocus every single time on a new tiles - which mean I increased the Zstack --> Another reason why I want to ran everything in OAD

I then realised that if I had to run it again on another set of specimens, it would failed and only restarting the program (I restart the computer to be sure) would help 'resetting' the memory and then it would work again. Do I need to add anything to the end of the workflow - OAD macro to clear everything ?

Finally, for SmartProof 5, if you wish to split scenes of a .czi image and save as SUR.
This is not currently possible in the software so need to be done in OAD

Code: Select all

imgin = Zen.Application.LoadImage('D:\\Tmp\\tmp.czi')
outpath = 'whereever'

for ii = range(1,imgin.Bounds.SizeS+1)
     imgspl =  Zen.Processing.Utilities.CreateSubset(imgin, 'S('+str(0)+','+str(ii)+')')
     Zen.Processing.Utilities.ExportHeightmapFromTopography(imgspl,outpath+'\\'+filename+str(ii)+'.sur')
No need to add - as seen elsewhere on this forum

Code: Select all

imgtop = Zen.Processing.Transformation.Topography.CreateTopography(imgspl,1,254) 
because the image is already a topography (also it give you an error if you try)
user-4
Posts: 398
Joined: Thu Jan 01, 1970 1:00 am

Post by user-4 »

I realised that some of the solutions provided are unique to light microscope, but does not apply to SmartProof
This is a forum for OAD on basis of Zen blue.
We also support Zen Core as far as LM (light microscope) is concerned.

But it is not easy for us to support SmartProof on ZenCore. Even if you had Zen Core in combination with LM it is necessarily to use Zen blue to generate a ‘valid experiment’ that you can load, modify and execute it in Zen Core.

I am trying to find someone who is in a position to give advice.
Carlton flores
Posts: 1
Joined: Mon Jun 13, 2022 9:34 am
Contact:

Re: SmartProof5 - Control Microscope for Surface Roughness (multiple locations across XY)

Post by Carlton flores »

I would recommend to have a close look at the Acquisition Tab and configure a Tiles-Experiment with tiles manually added as Positions.
CarlZeissMicroscopy3
Posts: 180
Joined: Wed May 20, 2020 10:10 am

Re: SmartProof5 - Control Microscope for Surface Roughness (multiple locations across XY)

Post by CarlZeissMicroscopy3 »

Hello Carlton flores,

as already stated above, this is a forum for Open Application Development (macro programming) with Zen blue.
Therefore this forum is not meant to discuss Zen specific features unless it has to do with OAD.

In Zen 'Tile Regions' are other 'objects' than 'Single Positions'.
Therefore 'Tile Regions' must stay 'Tile Regions', even if it just contains one tile, because the 'Tile Region object' (n the 'Tiles - Advanced Setup') allows you to change the size (e.g. from 1 Tile to 4X4 Tiles) whenever needed but a 'Single Positions' mainly can be moved.

I hope this answers your question.
Post Reply