Facilitating deep learning in Fortran with FTorch

Jack Atkinson

Principal Research Software Engineer
ICCS - University of Cambridge

Joe Wallwork

Senior Research Software Engineer
ICCS - University of Cambridge

2026-07-08

Precursors

Slides and Materials

To access links or follow on your own device these slides can be found at:
joewallwork.com/pwp/slides/ftorch/2026-07-08_DKRZ-Seminar

Licensing

Except where otherwise noted, these presentation materials are licensed under the Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) License.

Vectors and icons by SVG Repo under CC0(1.0) or FontAwesome under SIL OFL 1.1

Motivation

Weather and Climate Models

Large, complex, many-part systems.

Hybrid Modelling

Neural Net by 3Blue1Brown under fair dealing.
Pikachu © The Pokemon Company, used under fair dealing.

2 approaches: * emulation, or * data-driven.

Additional challenges: * Physical compatibility * Physics-based models have conservation laws
Required for accuracy and stability * Language interoperation

Challenges

• Reproducibility
  • Ensure net functions the same in-situ
• Re-usability
  • Make ML parameterisations available to many models
  • Facilitate easy re-training/adaptation
• Language Interoperation

Language interoperation

Many large scientific models are written in Fortran (or C, or C++).
Much deep learning is conducted in Python.

Mathematical Bridge by cmglee used under CC BY-SA 3.0
PyTorch, the PyTorch logo and any related marks are trademarks of The Linux Foundation.”
TensorFlow, the TensorFlow logo and any related marks are trademarks of Google Inc.

Emulation and data-driven is a focus of several of our projects. CALIPSO, M2LInES, DataWave, etc.
Models typically in Fortran
ML typically in Python

Some initial approaches to note

• Rewrite the model in a DL framework (e.g., JAX)
  • Advantages: avoids language inter-operation problem
  • Disadvantages: requires an enormous amount of developer effort

• Interface with Python ML via Forpy
  • Advantages: easy integration
  • Disadvantages: harder to use with ML and HPC, GPL, barely-maintained

• Fortran-Keras Bridge
  • Advantages: two-way coupling
  • Disadvantages: Keras only, abandonware

Approach 1: Transfer data between Python and Fortran

Notable package: SmartSim

Advantages:

• Generic, versatile, full control
• Two-way coupling
• HPC-friendly

Disadvantages

• Steep (human) learning curve
• Data copying

Approach 1: Transfer data

Python
env

Python
runtime

xkcd #1987 by Randall Munroe, used under CC BY-NC 2.5

Approach 2: Auto-generate inference code

Notable package: ENNUF

Advantages:

• Separation-of-concerns
• Low maintenance overhead

Disadvantages

• One-way coupling (inference only)
• Limited functionality (still early days)

Approach 3: Interface to C++ backend

Notable packages:

• TorchFort
• FTorch

Advantages:

• Inherit functionality and optimisations from a well-established package
• Two-way coupling with no data copying
• Avoid Python

Disadvantages

• Restricted to PyTorch

Approach 3: Interface to C++ backend

PyTorch has a C++ backend and provides an API.

Binding Fortran to C has been possible since 2003 via iso_c_binding.

If the user saves their PyTorch models in the portable TorchScript format then this can be used in C++.

By providing a Fortran API, FTorch and TorchFort wrap the libtorch C++ API, abstracting complex details from users.

TorchScript is a statically typed subset of Python

Approach 4: Implement DL in Fortran

Two notable packages:

• neural-fortran
• Fiats

Advantages:

• Avoids language inter-operation problem
• Allows full control from Fortran, e.g., performance optimisation

Disadvantages

• Reproducibility issues
• Hard for complex architectures

High-level comparison

Table taken from a draft review paper currently in development in collaboration between maintainers of the six packages.

The paper will guide the reader towards the best package for their use case.

FTorch

Efficiency

We consider 2 types:

Computational

Developer

In research both have an effect on ‘time-to-science’.
Especially when extensive research software support is unavailable.

1. No re-writing net after you have already done and trained
2. For scientists doing science, not compsci
3. Run on HPC environments so minimal additional
4. Easily keep up to date
5. Simple to learn and deploy
6. Needs to be as efficient as possible

Highlights - Developer

• Easy to clone and install
  • CMake, supported on linux/unix and Windows™

• Easy to link

  • Build using CMake,

  • or link via Make (instructions included)
    

    FCFLAGS += -I<path/to/install>/include/ftorch
    LDFLAGS += -L<path/to/install>/lib64 -lftorch

Find it on :

/Cambridge-ICCS/FTorch

Highlights - Developer

• User tools
  • pt2ts utility script aids users in saving PyTorch models to TorchScript

• Examples suite
  • Take users through full process from trained net to Fortran inference

• Full API documentation online at
  cambridge-iccs.github.io/FTorch

• FOSS
  • licensed under MIT
  • contributions from users via GitHub welcome

Find it on :

/Cambridge-ICCS/FTorch

Highlights - Computation

• Use framework’s implementations directly
  • feature and future support, and reproducible

• Make use of the Torch backends for GPU offload
  • CUDA, HIP, MPS, and XPU enabled

• Indexing issues and associated reshape¹ avoided with Torch strided accessor.
• No-copy access in memory (on CPU).

Find it on :

/Cambridge-ICCS/FTorch

1. Or, the great row-major, column-major headache.

Highlights - Computation

• Indexing issues and associated reshape¹ avoided with Torch strided accessor.
• No-copy access in memory (on CPU).

Find it on :

/Cambridge-ICCS/FTorch

1. Or, the great row-major, column-major headache.

Some code

Model - Saving from Python

import torch
import torchvision

# Load pre-trained model and put in eval mode
model = torchvision.models.resnet18(weights="IMAGENET1K_V1")
model.eval()

# Create dummmy input
dummy_input = torch.ones(1, 3, 224, 224)

# Save to TorchScript
if trace:
    ts_model = torch.jit.trace(model, dummy_input)
elif script:
    ts_model = torch.jit.script(model)
frozen_model = torch.jit.freeze(ts_model)
frozen_model.save("/path/to/saved_model.pt")

TorchScript

• Statically typed subset of Python
• Read by the Torch C++ interface (or any Torch API)
• Produces intermediate representation/graph of NN, including weights and biases
• trace for simple models, script more generally

Fortran

 use ftorch
 
 implicit none
 
 real, dimension(5), target :: in_data, out_data  ! Fortran data structures
 
 type(torch_tensor), dimension(1) :: input_tensors, output_tensors  ! Set up Torch data structures
 type(torch_model) :: torch_net
 integer, dimension(1) :: tensor_layout = [1]
 
 in_data = ...  ! Prepare data in Fortran
 
 ! Create Torch input/output tensors from the Fortran arrays
 call torch_tensor_from_array(input_tensors(1), in_data, torch_kCPU)
 call torch_tensor_from_array(output_tensors(1), out_data, torch_kCPU)
 
 call torch_model_load(torch_net, 'path/to/saved/model.pt', torch_kCPU)  ! Load ML model
 call torch_model_forward(torch_net, input_tensors, output_tensors)      ! Infer
 
 call further_code(out_data)  ! Use output data in Fortran immediately
 
 ! Cleanup
call torch_delete(model)
call torch_delete(in_tensors)
call torch_delete(out_tensor)

GPU Acceleration

Cast Tensors to GPU in Fortran:

! Load in from TorchScript
call torch_model_load(torch_net, 'path/to/saved/model.pt', torch_kCUDA, device_index=0)

! Cast Fortran data to Tensors
call torch_tensor_from_array(in_tensor(1), in_data, torch_kCUDA, device_index=0)
call torch_tensor_from_array(out_tensor(1), out_data, torch_kCPU)

FTorch supports NVIDIA CUDA, AMD HIP, Intel XPU, and AppleSilicon MPS hardwares.

Use of multiple devices supported.

Effective HPC simulation requires MPI_Gather() for efficient data transfer.

Through developing a nice library it behaves as you would expect it to!
Familiar to pytorch users.
Pointer slinging is largely hidden from users.
Data transposition is taken care of by us (but options available for users to handle).

Publication & tutorials

FTorch is published in JOSS!

Atkinson et al. (2025)
FTorch: a library for coupling PyTorch models to Fortran.
Journal of Open Source Software, 10(107), 7602,
doi.org/10.21105/joss.07602

Please cite if you use FTorch!

In addition to the comprehensive examples in the FTorch repository we provide an online workshop at /Cambridge-ICCS/FTorch-workshop

Applications and Case Studies

MiMA - proof of concept

• The origins of FTorch
  • Emulation of existing parameterisation
  • Coupled to an atmospheric model using forpy in Espinosa et al. (2022)¹
  • Prohibitively slow and hard to implement
  • Asked for a faster, user-friendly implementation that can be used in future studies.

• Follow up paper using FTorch: Uncertainty Quantification of a Machine Learning Subgrid-Scale Parameterization for Atmospheric Gravity Waves (Mansfield and Sheshadri 2024)
  • “Identical” offline networks have very different behaviours when deployed online.

Model of an idealised Moist Atmosphere online at /DataWaveProject/MiMA-machine-learning

1. Originally written in TensorFlow…

ICON

• Icosahedral Nonhydrostatic Weather and Climate Model
  • Developed by DKRZ
  • Used by the DWD and Meteo-Swiss

• Interpretable multiscale Machine Learning-Based Parameterizations of Convection for ICON (Heuer et al. 2023)¹
  • Train U-Net convection scheme on high-res simulation
  • Deploy in ICON via FTorch coupling
  • Evaluate physical realism (causality) using SHAP values
  • Online stability improved when non-causal relations are eliminated from the net

Running on JUWELS at the Jülich Supercomputing Centre; ICON Model is available at gitlab.dkrz.de/icon/icon-model

1. Work without direct involvement of ICCS - success on ‘ease of use’ of FTorch

ICON

Slide from the Cambridge ML Coupling Workshop courtesy of Julien Savre

SuperdropNet

Slide from the Cambridge ML Coupling Workshop courtesy of Caroline Arnold and Paul Keil

CESM coupling

• The Community Earth System Model
• Part of CMIP (Coupled Model Intercomparison Project)
• Make it easy for users
  • FTorch integrated into the build system (CIME)
    • /Cambridge-ICCS/cime_je
  • libtorch is included on the software stack on Derecho
    • Improves reproducibility

Derecho by NCAR

Work done with Will Chapman and Jim Edwards of NCAR.
CIME with FTorch is available with instructions /Cambridge-ICCS/cime_je

Others

• To replace a BiCGStab bottleneck in the GloSea6 Seasonal Forecasting model
  (Park and Chung 2025).
• Bias correction of CESM through learning model biases compared to ERA5
  (Chapman and Berner 2025)
• Implementation of nonlinear interactions in the WaveWatch III model
  (Ikuyajolu et al. 2025).
• Stable embedding of a convection resolving parameterisation in E3SM
  (Hu et al. 2025).

• ClimSim Convection scheme in ICON for stable 20-year AMIP run
  (Heuer et al. 2025) (preprint)
• Review paper of hybrid modelling approaches
  (Zheng et al. 2025) (preprint)
• Implementation of a new convection trigger in the CAM model.
  Miller et al. In Preparation.
• Embedding of ML schemes for gravity waves in the CAM model.
  ICCS & DataWave.

Recent updates

v1.0 Contributors

10 Total, 5 new!

• Jack Atkinson, Joe Wallwork - Maintainers
• Mikolaj Kowalski, Tom Meltzer - ICCS RSEs
• Niccolò Zanotti - Placement student
• Jared Frazier, Zhenkun Li, Zoltán Katona - Community Users
• Dominic Orchard, Daniel Katz - code and paper review

What’s New - v1.1

• Simplify torch_tensor_from_array signature.
• Implement finalizers for torch_tensor and torch_model.
• Added documentation and examples clarifying how to use batching.
• Improved error handling.
• AMD GPU support (HIP).
• pkg-config support.
• Option to build as static library.
• Comprehensive unit testing with pFUnit.
• Extended CI pipeline to cover MacOS and Windows, Intel compilers, and to run on GPU.
• Documentation overhaul (cambridge-iccs.github.io/FTorch)

What’s New - v1.2

• Online training
  • Exposed autograd functionality from Torch.
  • torch_optim derived type wrapping PyTorch optimizers
  • torch_loss_mse and torch_loss_cross_entropy loss subroutines.
  • Full Fortran training loop now demonstrated in worked example 11
• utils is now ftorch_utils, installable via pip
• Overhauled pt2ts approach, moving from template code to a command-line script.
• Extended CI pipeline to cover Flang compiler.

ML Coupling Workshop

ICCS ran a two-day ML Coupling Workshop in Cambridge, September 2025, bringing together researchers, RSEs, and modelling centres working on hybrid modelling challenges.

• Shared recent advances and expertise in coupling ML to large-scale scientific codes
• Discussion sessions on key challenges and best practices
• Summary blog post: Accelerate-C2D3

Ongoing and future work

• Further compiler/CI support:
  • LFortran
  • nvfortran
• Package and distribute via fpm (Fortran Package Manager)
• Summer intern working on UKCA case study.
• Ongoing inter-comparison study.

Join the FTorch mailing list for updates!

Your Project Here?

FTorch: Summary

• Use of DL within traditional numerical models
  • A growing area that presents challenges
• Language interoperation
  • FTorch provides a solution for scientists implementing torch models in Fortran
  • Designed for computational and developer efficiency
  • Has helped deliver science in climate research and beyond
    See FTorch/community/case_studies
  • Built into CESM to allow the userbase access
• Lots of improvements recently merged!

Fortran-Enzyme

Automatic differentiation for Fortran, via Enzyme

Compiler plugin that differentiates LLVM IR, computing gradients of existing code without source-level rewriting. Currently supports C, C++, Julia, Rust. ICCS are working to add Fortran.

6-month (March-Aug) ICCS project.

Aiming to provide Fortran bindings for Enzyme, tests, and documentation and to demonstrate differentiable Fortran on a materials science case study.

Eventual goal is to provide a simple tool for differentiable modelling of existing Fortran code.

/EnzymeAD/Enzyme

Thanks for Listening

Get in touch:

 Jack Atkinson

 jackatkinson.net

 jwa34[AT]cam.ac.uk

 jatkinson1000

 @jatkinson1000@hachyderm.io

 Joe Wallwork

 joewallwork.com

 jw2423[AT]cam.ac.uk

 joewallwork

Thanks to Tom Meltzer, Elliott Kasoar, Niccolò Zanotti
and the rest of the FTorch team.

The ICCS received support from

FTorch has been supported by

/Cambridge-ICCS/FTorch

References

Atkinson, Jack, Athena Elafrou, Elliott Kasoar, Joseph G. Wallwork, Thomas Meltzer, Simon Clifford, Dominic Orchard, and Chris Edsall. 2025. “FTorch: A Library for Coupling PyTorch Models to Fortran.” Journal of Open Source Software 10 (107): 7602. https://doi.org/10.21105/joss.07602.

Chapman, William E, and Judith Berner. 2025. “Improving Climate Bias and Variability via CNN-Based State-Dependent Model-Error Corrections.” Geophysical Research Letters 52 (6): e2024GL114106. https://doi.org/10.1029/2024GL114106.

Espinosa, Zachary I, Aditi Sheshadri, Gerald R Cain, Edwin P Gerber, and Kevin J DallaSanta. 2022. “Machine Learning Gravity Wave Parameterization Generalizes to Capture the QBO and Response to Increased CO2.” Geophysical Research Letters 49 (8): e2022GL098174.

Heuer, Helge, Tom Beucler, Mierk Schwabe, Julien Savre, Manuel Schlund, and Veronika Eyring. 2025. “Beyond the Training Data: Confidence-Guided Mixing of Parameterizations in a Hybrid AI-Climate Model.” arXiv Preprint arXiv:2510.08107. https://doi.org/10.48550/arXiv.2510.08107.

Heuer, Helge, Mierk Schwabe, Pierre Gentine, Marco A Giorgetta, and Veronika Eyring. 2023. “Interpretable Multiscale Machine Learning-Based Parameterizations of Convection for ICON.” arXiv Preprint arXiv:2311.03251.

Hu, Zeyuan, Akshay Subramaniam, Zhiming Kuang, Jerry Lin, Sungduk Yu, Walter M Hannah, Noah D Brenowitz, Josh Romero, and Michael S Pritchard. 2025. “Stable Machine-Learning Parameterization of Subgrid Processes in a Comprehensive Atmospheric Model Learned from Embedded Convection-Permitting Simulations.” Journal of Advances in Modeling Earth Systems 17 (7): e2024MS004618.

Ikuyajolu, Olawale James, Luke P Van Roekel, Steven R Brus, and Erin E Thomas. 2025. “NLML: A Deep Neural Network Emulator for the Exact Nonlinear Interactions in a Wind Wave Model.” Authorea Preprints. https://doi.org/10.22541/essoar.174366388.80605654/v1.

Mansfield, Laura A, and Aditi Sheshadri. 2024. “Uncertainty Quantification of a Machine Learning Subgrid-Scale Parameterization for Atmospheric Gravity Waves.” Authorea Preprints.

Park, Hyesung, and Sungwook Chung. 2025. “Utilization of a Lightweight 3D u-Net Model for Reducing Execution Time of Numerical Weather Prediction Models.” Atmosphere 16 (1): 60.

Zheng, Tian, Subashree Venkatasubramanian, Shuolin Li, Amy Braverman, Xinyi Ke, Zhewen Hou, Peter Jin, and Samarth Sanjay Agrawal. 2025. “Machine Learning Workflows in Climate Modeling: Design Patterns and Insights from Case Studies.” arXiv Preprint arXiv:2510.03305. https://doi.org/10.48550/arXiv.2510.03305.

What’s New - v1.1 (detailed)

No need to specify layout

Previously when calling torch_tensor_from_array one had to specify the memory layout. This was used to correctly stride in memory to avoid copying.

We now assume users want the [1, 2, ..., n] layout by default, with layout not needing to be specified.

Older code continues to work in v1.1, but in future layout will become an optional argument requiring a change in order of call arguments.

Advice: Use the default argument where possible.

Pull request #348

Finalisers for tensors and models

FTorch creates Torch C++ objects for manipulation under the hood. As detailed in the documentation and examples, proper handling is required, like allocate and deallocate to prevent memory leakage.

Previously one had to explicitly delete torch objects to ensure that C++ memory was cleaned up, otherwise leakage could occur.

Now torch_delete is a finalizer, meaning it will be called whenever a tensor, model, or array of tensors goes out of scope.

Continuing to call torch_delete will still work, so old code remains valid.

Pull Request #297

Finalisers - Code Comparison

Before: explicit cleanup required

use ftorch

implicit none

real, dimension(5), target :: in_data, out_data

type(torch_tensor), dimension(1) :: input_tensors, output_tensors
type(torch_model) :: torch_net

...

! Create Torch input/output tensors from the Fortran arrays
call torch_tensor_from_array(input_tensors(1), in_data, torch_kCPU)
call torch_tensor_from_array(output_tensors(1), out_data, torch_kCPU)

call torch_model_load(torch_net, 'path/to/saved/model.pt', torch_kCPU)
call torch_model_forward(torch_net, input_tensors, output_tensors)

...

! Cleanup
call torch_delete(torch_net)
call torch_delete(input_tensors)
call torch_delete(output_tensors)

After: finalizer handles cleanup

use ftorch

implicit none

real, dimension(5), target :: in_data, out_data

type(torch_tensor), dimension(1) :: input_tensors, output_tensors
type(torch_model) :: torch_net

...

! Create Torch input/output tensors from the Fortran arrays
call torch_tensor_from_array(input_tensors(1), in_data, torch_kCPU)
call torch_tensor_from_array(output_tensors(1), out_data, torch_kCPU)

call torch_model_load(torch_net, 'path/to/saved/model.pt', torch_kCPU)
call torch_model_forward(torch_net, input_tensors, output_tensors)

...

Finalizers - Aside

torch_tensor_delete has been made elemental, meaning that it applies to both tensors and arrays of tensors.

As such torch_tensor_array_delete has been removed.

This shoud not affect users as advice has always been to use the torch_delete interface instead of calling directly.

Pull request #545

Clarification on Batching

Batching works the same as in PyTorch — add a leading batch dimension to input arrays and FTorch applies the model independently to each element.

This has always been the case, but we have clarified this in the documentation and added a worked example - 04) Batching.

Pull Request #500

Key points:

• Leading dimensions are batch size, Trailing dimensions must match the model’s expected feature size.
• All input band output tensors must share the same batch dimensions (pre-allocated).
• One model can handle both single and batched inference.

Clarification on Batching

! Single inference (1D input)
real(sp), dimension(5), target :: in_single, out_single

! Batched inference (3D input)
real(sp), dimension(2,3,5), target :: in_batch, out_batch

call torch_model_load(model, "model.pt", torch_kCPU)

! Single
call torch_tensor_from_array(in_tensors(1), in_single, torch_kCPU)
call torch_tensor_from_array(out_tensors(1), out_single, torch_kCPU)
call torch_model_forward(model, in_tensors, out_tensors)

! Batched
call torch_tensor_from_array(in_tensors(1), in_batch, torch_kCPU)
call torch_tensor_from_array(out_tensors(1), out_batch, torch_kCPU)
call torch_model_forward(model, in_tensors, out_tensors)

Improved Error Handling

Previously errors in Torch resulted in opaque error messages that referenced locations in the Torch library at the point of failure.

Now there are checks in Fortran to catch some input errors before they propagate to the C++ and raise them there, and C++ errors are now caught and handled in the CTorch layer to provide more information to the user about the point of failure.

Before: opaque Torch C++ exception

terminate called after throwing
  an instance of 'torch::Error'
  what(): Expected all tensors to
  be on the same device, but found
  at least two devices (cuda:0
  and cpu)!

After: caught at CTorch layer

[ERROR]: One of the inputs to torch_jit_module_forward is not a Tensor

Pull request #347

No passing of temporaries

Internally, FTorch makes the assumption that data passed in is contiguous. in memory. This is due to the shared memory feature for efficiency.

If this is violated then data could be read incorrectly by Torch.

Validation is now applied in torch_tensor_from_array to check that input data is pointer, contiguous rather than simply target.

This is considered a bugfix but if you were passing in temporaries you will now need to create an array first.

The following calls are now forbidden:

! slice — non-contiguous subsection
call torch_tensor_from_array(t, data(1:5:2), torch_kCPU)

! array expression — compiler-generated temporary
call torch_tensor_from_array(t, in_data * 2, torch_kCPU)

! function result — unnamed temporary
call torch_tensor_from_array(t, get_array(), torch_kCPU)

No passing of temporaries

What do you need to do?

Nothing, provided you always passed arrays into torch_tensor_from_array.

How shall ye know it?

The following error will be raised in compilation:

Error: There is no specific subroutine for the generic 'torch_tensor_from_array' at (1)

with the 1 identifying the input data argument.

AMD GPU Support (HIP)

FTorch now supports AMD GPUs via the HIP backend.

PyTorch recommends reusing torch.cuda interfaces for HIP — FTorch builds against the CUDA backend, aliasing HIP at the CMake level.

! Same interface, compiled with HIP flags
call torch_model_load(torch_net, 'model.pt', torch_kHIP, device_index=0)
call torch_tensor_from_array(in_tensor(1), in_data, torch_kHIP, device_index=0)

Pull request #385 and Pull request #388

Source code restructure

Since the early days all FTorch source code existed in src/ftorch.F90. As features grew this became unsustainable so it is now distributed across several files:

src
├── ctorch.cpp
├── ctorch.h
├── ftorch_devices.F90
├── ftorch_model.f90
├── ftorch_optim.f90
├── ftorch_tensor.f90
├── ftorch_tensor.fypp
├── ftorch_types.f90
└── ftorch.f90

No change to usage — users still import from a parent ftorch module.

Sysadmin

• CMake 3.18 — minimum version bumped to match PyTorch PR #491
• pkg-config — query compilation flags via pkg-config --libs ftorch and pkg-config --cflags ftorch PR #464
• Static library — support building as a static library PR #448
• RPATH — libftorch.so now includes RUNPATH to Torch so downstream targets find it automatically PR #437

Thanks to Jared Frazier from Leibniz Institute of Atmospheric Physics for community contributions.

Sustainability/Other

• Comprehensive unit testing with pFUnit
  • All new contributions to the software came with extensive unit testing to provide users with confidence.
  • New features add integration tests and examples.
• Extensive CI pipeline to catch issues and provide broad support.
  • Linux, macOS, Windows
  • GNU and Intel compilers
  • CPU and GPU backends
• Documentation overhaul
  • cambridge-iccs.github.io/FTorch
  • Improved aesthetics
  • Tidied user pages and API docs, added community pages (PRs welcome)

What’s New - v1.2 (detailed)

Online training - Autograd

• Exposed autograd functionality from Torch
  • requires_grad argument on tensor construction
  • torch_tensor_backward for reverse-mode differentiation
  • torch_tensor_get_gradient to extract computed gradients
  • torch_tensor_zero_grad to reset gradients between backward passes
• Mathematical operator overloading (=, +, -, *, /, **)
  • Enables tensor expressions in Fortran that build a computation graph
  • Further expressions can be added - requests/PRs welcome

Online training - Optimizers

• torch_optim derived type wrapping PyTorch optimizers
  • torch_optim%zero_grad — zeroes gradients at start of each step
  • torch_optim%step — takes one optimizer iteration
• Optimizers exposed: SGD, Adam, AdamW
• Others can be added - requests/PRs welcome

Testing optimisation of a single tensor against PyTorch:

Online training - Loss

• torch_loss_<...> subroutines create loss tensors.
• Loss functions exposed: MSE and CrossEntropy
• Others can be added - requests/PRs welcome

! Old calculation
call torch_tensor_mean(loss, (output_vec - target_vec) ** 2)

! Now with loss subroutine
call torch_loss_mse(loss, output_vec, target_vec)

Online training - Model training

• Full Fortran training loop now demonstrated in worked example 11
  • Load a TorchScript model, train using an optimizer, and run inference
• torch_model_parameters interface added to access model weights
• Loss curves comparable to equivalent PyTorch training.

Training SimpleNet in Fortran:

Overhaul of pt2ts

• utils is now ftorch_utils, installable via pip
• pt2ts is now a command-line script

  • No longer required to copy and modify script by hand.

  • provide model definition file, the class name, and where to save:

    pt2ts SimpleNet --model_definition_file simplenet.py \
                    --output_model_file model.pt

Pull request #555