Skip to content

Ansys Lumerical simulations in Calcula

If you're looking forward to working with Ansys Lumerical, first you will need to follow some steps to launch the program. Then, we will have to configure it so that it computes all data in the Calcula powerful servers while still working in your directories. If you need help with the Licence, please reach the IT services of the department and they will help you with everything you need :)

1. Opening Ansys Lumerical

To do so, first Open a new terminal by going to Menu -> System Tools -> LXTerminal. Here is where we will write the instructions and commands to open Lumerical. After each command in the LXTerminal (not in Lumerical) you write, always press ENTER to submit it.

 

Setting up ssh (Only the first time).

If it's your first time trying to open Lumerical, follow these two steps before continuing to the standard procedure.

These commands help to set up ssh service, and only need to be made the first time: Press Enter in all questions and then execute the second command.

ssh-keygen # Press enter in all questions!
cat .ssh/id_rsa.pub >> .ssh/authorized_keys # All the line is one command only

From now on, the procedure is standard and should be followed from here the next time you open Lumerical.


Standard procedure to launch Lumerical

Once the terminal is open, we can write the first command that will help activate the graphic parts of Lumerical:

>> ssh -X localhost

After this step is done, we are finally ready to launch Lumerical. Simply type it on the command line and it will open automatically:

>> lumerical & 

2. Work with Ansys Lumerical

Once inside the program, you can freely use any solver or example to work and develop your project.

After you are finished and want to check results, instead of working with "your own computer", you can use the power of Calcula.

3. Simulating with Calcula power

Before running your simulation, and the aim of this guide, it's important to configure the resources that will be used in order to perform the computations.

First of all, we need to open the Simulations section in the settings bar and click on Configure resources (if this top bar doesn't appear, it is located in file -> resources). Here, we will define that we want to use calcula servers' power.

Resources_bar.jpg

Then, depending on the solver you are using you will need to do different things, but, in general, they are all configured the same way.

After clicking on the configure resources section a window will pop with different solver configurations.

It is crucial to configure EVERY solver, otherwise, you will use your less powerful computer and simulations will be very slow.

config_window.JPG

In this new window, for EACH SOLVER:

  • 1. Change Name: Doble click on "Local Host" and change it to "Calcula". Press enter.
  • 2. Configure Solver: if the solver is FDTD, see section A. Otherwise, see section B.

You DO NOT NEED to configure the Design environment window.

Once you have configured every solver, run the simulation by clicking on the Run icon.

run.jpg

There are 5 licenses available for running simulations. These licenses are shared so, if nobody is running one, you can have up to 5 simulations running at the same time (from different work projects).

It is important to understand that the task of simulating your work will be queued in the sunset servers. This means that, until the resources (or licenses) you asked are not free, the simulation will remain on pause. So don't worry if it doesn't compute instantaneously, it is probably because you will have to wait.

 

A. FDTD solver

In the solver window, click on Edit to edit ("Calcula") to change its settings. A new window will pop.

First, change Job launching preset to Job Scheduler: Slurm.
Then, on the Job Scheduler Option section, you will need to change both the command and Submission script.

  • Command section: Here we define to use the sunset calcula servers and the resources used to run the simulations. In this case 1 core with 5 tasks per core and 2Gb of RAM memory. If you need more computational power, check the last section.
sbatch -p sunset -A sunset -c1 --mem=2G -N 1 --ntasks-per-node=5
  • Submission script: In this section, erase the initial code and just copy and paste the following one instead of the pre-defined one.
#!/bin/bash
source /opt/intel/oneapi/setvars.sh --force
mpirun /opt/lumerical/v211/bin/fdtd-engine-impi-lcl -logall -remote {PROJECT_FILE_PATH}

After all changes are done, press OK to save them.

If you have more solvers in the Resource configuration window, check section B to set them up the right way (don't forget to change the Name of the "Local Host" to "Calcula"!).

Then, press Save to correctly configure Resources and you are ready to Run your simulations. They will be stored in your directory, but computed in the calcula servers.

 

B. Other solvers

This section includes the following solvers: Eigensolver, CHARGE, HEAT, DGTD, FEEM…

Here, just like in section A, everything starts by clicking on Edit to edit the ("Calcula") settings.

Then, change Job launching preset to Job Scheduler: Slurm and change the Command section.

  • Command:  In this case, we are asking for 4 cores and 1Gb of memory to run our simulation. If you need more power (processors or RAM memory), check the last section of the guide.
sbatch -p sunset -A sunset -c4 --mem=1G

After this change is done, press OK to save it.

If you have more solvers in the Resource configuration window, check section A or B to set them up the right way (don't forget to change the Name of the "Local Host" to "Calcula"!).

Then, press Save to correctly configure Resources and you are ready to Run your simulations. Before simulating, always check this is configured so that you are using calcula server’s power.

Using more resources for more demanding projects

If you want to use more resources because simulations are too slow, you can edit some parameters in the resources window to help run your simulations faster. Each solver (type A or B) is edited in a different way.

Check the landing page to see the capability of the sunset servers (cores and RAM memory available).


A. FDTD Solver

With the resources window open, you can edit the command section to increment the amount of resources used.

  • Command: In this case, you can change the parameter "-cN" and "--ntasks-per-node=M". The multiplication of N*M needs to be smaller or equal than 16. It is suitable to maintain "N" low and "M" high. For example: "-c2" and "--ntasks-per-node=8" or even "-c1" and "--ntasks-per-node=16".
sbatch -p sunset -A sunset -cN --mem=2G -N 1 --ntasks-per-node=M

You can also increase RAM memory used by changing the “--mem” parameter to, for example, 4 Gb (or even more if you need it).

--mem=4G


B. Other Solvers

With the resources window open, you can edit the command section to increment the number of cores used up to N cores.

  • Command (set N from 1 to 16): N defines the number of cores used. If you ask for many cores, you might have to wait for the simulation to run before all cores are ready for you (they may be some of them already in use by another user).
sbatch -p sunset -A sunset -cN --mem=1G

You can also increase RAM memory used by changing the “--mem” parameter to, for example, 4 Gb (or even more if you need it).

--mem=4G