NUBEAM

1. Code Name: NUBEAM
2. Code Category: Neutral beam deposition, beam fast ions, fusion product fast ions.
3. Primary Developer: D. McCune & R. Goldston (originally); M. Gorelenkova at present.
4. Other Developers and Users: Developers in CPPG: Rob Andre (enhanced FLR model for STs); K. Indireshkumar (past development on NUBEAM MPI features), Marina Gorelenkova (3d halo model, efficiency improvements, I/O improvements, MPI improvements, componentization for SWIM FACETs w/Doug McCune); Other developers: Alexei Pankin (then at Lehigh University) as part of NTCC project in early 2000s Ð 1st separation of NUBEAM from TRANSP allowing it to be used as a separate library ("NTCC module") and later as a SWIM SciDAC component; this collaboration also led to Pankin's 2004 CPC article which gives the best available published description of the code. Users: almost all TRANSP users are NUBEAM users. In addition, NUBEAM is now coupled to SWIM and FACETs SciDACs. The NTCC version has been coupled and is actively used in ONETWO at General Atomics. It has also been coupled to and used in ASTRA by scientists at JET and elsewhere in the EU (we think).
5. Short description (one line if possible): direct simulation time dependent kinetic Monte Carlo beam deposition and fast ion orbit & slowing down model in 2d axisymmetric tokamak geometry; it is also extensively used for fusion product fast ions.
6. Computer Language (Fortran77, Fortran90, C, C++, etc) and approx # of lines: Fortran-90, about 100k lines of code, in NUBEAM itself; NTCC libraries used by NUBEAM encompass another 200k lines of code approximately.
7. Type of input required (including files and/or output from other codes). Is there any special input preparation system (eg, GUI)? The NUBEAM subroutine library or free standing component is meant to be attached to a time dependent transport code or tokamak whole device model. Its main input (as recently recoded) is a Plasma State object containing the neutral beam machine description (geometry) and target plasma composition and parameters. An additional namelist controls numerical options e.g. the number of Monte Carlo particles to use per specie, and physics modeling choices e.g. PREACT or ADAS for atomic reaction rate data.
8. Type of output produced (including files that are read by other codes and sizes of large files and synthetic diagnostics): Much of NUBEAMs output (i.e. heating and current drive profiles and fast ion densities and pressure components) are written back to the Plasma State object that was provided as input. Many additional details can be retrieved through specialized memory or file data structures: fast ion distribution functions vs. (Psi, theta, E_lab, vpll/v), 3d deposition source profiles, orbit-lost particle distributions, ...
9. Describe any postprocessors which read the output files: there is a generic Plasma State visualization tool (cstate). For ion or ion source distribution functions get_fbm is widely used for visualization, to produce NetCDF representations of the data, and/or to produce particle lists through random sampling.
10. Status and location of code input/output documentation: the main description document for namelist and code use is under NUBEAM at the catalog kept at http://w3.pppl.gov/NTCC; this is automatically updated from the TRANSP source repository which includes NUBEAM. The code web site also links to related documents and publication lists. While others (e.g. Holger St. John, the ONETWO author at General Atomics) have succeeded in using this documentation to implement their own NUBEAM deployments, it is not easy. The new Plasma State interface has helped considerably (but it is still not easy; the code physics capabilities are detailed and extensive).
11. Code web site: http://w3.pppl.gov/ntcc/NUBEAM/ -- actively maintained, with links to all available user and input/output documentation.
12. Is code under version control? What system? Is automated regression testing performed? The source code is under svn control, hosted on svnsrv.pppl.gov, with on- and off-site developer access allowed; it is contained within the TRANSP svn repository. At present, however, there is no mechanism for receiving NUBEAM (100k lines) without receiving the whole rest of TRANSP (2.4M lines including NUBEAM).
13. One to two paragraph descriptions of equations solved and functionality including what discretizations are used in space and time: Beam deposition: 3d straight line tracking through axisymmetric flux surface geometry, with deposition profiles calculated from charge exchange and impact ionization rate data. Beam orbiting: guiding center drift orbit equations, either in flux coordinates or (2x slower computationally) in (R,Z) coordinates, in presence of prescribed magnetic field and radial electrostatic field. FLR effects: fast simple model assumes r_larmor << grad(B) scale length; a much more expensive model for STs can avoid this but then dominates runtime. Collision operator: as described in Goldston's 1981 JCP paper with modest improvements: Fokker Planck model for slowing down, pitch angle scattering, and energy diffusion. Charge exchange loss and recapture: Monte Carlo method using the same atomic rate data as was needed for deposition; effects of charge exchange of beam fast ions with beam neutrals are estimated and are important in low density cases. Fusion rates and neutron source: Monte Carlo integration during orbiting for beam-target fusion; Monte Carlo convolution integral over accumulated distribution function, at end of model time step, for beam-beam fusion. 4d ion distribution functions (2d axisymmetric in space, 2d in velocity) can be saved; 3d deposition ion source distributions for each energy fraction of each neutral beam can also be saved. The Monte Carlo model follows fast ions only using classical physics + an anomalous radial diffusion operator with user provided diffusivities if requested; only fast ions are followed; when ions reach local (3/2) Ti they are declared thermal and turned over to the plasma model in the form of a Monte Carlo summed "thermalization" source function.
14. What modes of operation of code are there (eg: linear, nonlinear, reduced models, etc): The code always runs as a Monte Carlo kinetic particle model against a prescribed plasma target and field. For time dependence the plasma target and MHD equilibrium can change from step to step, with saved Monte Carlo lists of partially slowed down fast ions transformed to conserve their toroidal flux location, energy, and canonical angular momentum.
15. Journal references describing code: Standard references: R.J. Goldston et al., "New Techniques for Calculating Heat and Particle Source Rates due to Neutral Beam Injection in Axisymmetric Tokamaks", J. Comp. Phys. 43 (1981) 61; A. Pankin et al., "The Tokamak Monte Carlo Fast Ion Module NUBEAM in the National Transport Code Collaboration Library", Computer Physics Communications Vol. 159, No. 3 (2004) 157-184.
16. Codes it is similar to and differences (public version): (There is a recent paper benchmarking beam codes... I will review this to fill this section in - DMC).
17. Results of code verification and convergence studies (with references): (I will fill this in after making a query to users -DMC).
18. Present and recent applications and validation exercises (with references as available). (I will fill this in after making a query to users - DMC).
19. Limitations of code parameter regime (dimensionless parameters accessible): NUBEAM is a classical model; except for a sawtooth mixing model there is generally no description of MHD modes on fast ion confinement. There is plenty of experimental evidence that TAE modes affect fast ion confinement under some conditions; the code's "anomalous radial diffusion" only provides a crude treatment for such effects. Also, the "goosing method" (exploiting time scale separation between orbit bounce and collision time scales to accelerate the collision operator and reduce the time spent orbiting) deeply assumes axisymmetric geometry, so there is likely no efficient upgrade path for supporting stellarator or perturbed tokamak 3d field MHD equilibrium. The collision operator has no fast ion on fast ion collision term. There is strong experimental evidence of coupling of ICRF wave power to beam injected fast ions; there is a SciDAC research effort, specifically targeted at a NUBEAM upgrade, to address this, but it is not available yet.
20. What third party software is used? (eg. Meshing software, PETSc, ...): NetCDF, MDS+, lapack/blas.
21. Description of scalability: It's an "embarrassingly parallelizable" particle code. The code includes an MPI capability and scales well, even on Linux clusters with modest-cost interconnects. On a 1Gbe interconnect, weak scaling is good with problem sizes of 4000 MC particles per cpu thread. On an infiniband or stronger interconnect a somewhat smaller particle count gets good weak scaling. The SWIM SciDAC has run 1000000 particle NUBEAM simulations on 1000p at NERSC. However, for most NUBEAM applications (e.g. to produce 1d heating profiles for transport codes) a large particle list size is not needed.
22. Major serial and parallel bottlenecks: The deposition and orbiting particle loops are fully MPI-parallelized. However, there are other tasks (preparing input data, setting up field splines for orbiting and setting up total stopping rates for beam deposition; after orbiting, MPI-reduction of summed outputs) that represent significant serial overhead. It is possible that the code could obtain better scaling for small problem sizes, if optimization of NUBEAM data pre- and post-processing were optimized or (where feasible) parallelized. It would be a lot of work with no guarantee of benefit, because there are a lot of small tasks to be looked at. A major upgrade is underway to imbed within each neutral beam deposition region a 3d neutral halo model, for diagnostic simulation purposes. This is expected to be computationally expensive and will require MPI parallelization.
23. Are there smaller codes contained in the larger code? Describe: interpolation libraries (plasma state, xplasma, pspline); NetCDF for I/O; atomic physics packages.
24. Supported platforms and portability: this is mainly a question of fortran compilers. The code generally runs well on systems using Pathscale fortran, Intel fortran, Lahey-Fujitsu fortran, gfortran. There have been problems running over PGI fortran and IBM fortran; these could be overcome if there is ever sufficient reason to do so.
25. Illustrations of time-to-solution on different platforms and for different complexity of physics, if applicable: A single NUBEAM step (looping over all particles) can usually be completed in 20-60 seconds. Some features (R,Z orbiting, ST extended FLR model) can add substantially to the cost of a time step. Generally, NUBEAM does not need to be tightly coupled to a transport model, and often it is possible to take quite long time steps, possibly much longer than the transport code's internal time step for equilibrium and/or profile advance. In this case, NUBEAM output heating and current drive profiles and momentum and particles sources are treated by the transport model as step functions in time. For fast ion depletion and MHD pressure contributions, the data from the current and prior step need to be retained and piecewise linear interpolation applied to enforce continuity in time.
26. Any additional comments: For many years, NUBEAM was only accessible through TRANSP. The NTCC modularization and subsequent SWIM SciDAC componentization with Plasma State interface were successful in separating NUBEAM from TRANSP and making it a useful resource for any future transport code, SciDAC or FSP.