Empire AI Monthly User Session: September 2026
Transcript
Good afternoon, everyone, and thank you for joining our first Columbia Empire AI User Session. My name is Elizabeth Kwon, and I am the Senior Research Computing Specialist on the Research Computing Services team at CYT, and the primary Columbia Technical Point of Contact, and I'll be your host today.
This session is the first of a series that we've started for the Columbia Empire AI research community. Some general housekeeping before we begin. If you have a question throughout the presentation, feel free to drop it in the chat, and someone on our team will get to it, or hold it for the Q&A block at the end.
We'll also be recording today's session, so please meet yourselves and turn off your video, and a recording will be made available on our Research Computing Services video library later on.
We'll also share out the slides and the follow-up email communication. So, with that out of the way, let's get into it.
Looking at today's agenda, we'll cover who we are, what our team supports, a quick overview of Empire AI, walk through how to navigate the Empire AI resources, share a new funding opportunity, review feedback we have received so far from the user community, and leave room at the end for open Q&A.
Who we have here are the Columbia Technical Points of Contact. We are the team you'll interact with most directly for anything Empire AI-related on the Columbia side. Alongside me, we have Al, Waqas, and Halyna, our Senior Research Systems Engineers who handle day-to-day technical support for Empire AI, in addition to our on-prem HPC clusters.
The Columbia Technical Points of Contact sit within the Research Computing Services Department at CUIT, and we actively collaborate with the Office of the Executive Vice President for Research.
Empire AI itself is a separate multi-institutional consortium. We're one of eight member institutions alongside Cornell, NYU, the CUNYs, the SUNYs, and other institutions. Each institution also has their own team of institutional technical points of contact.
And our team is Columbia's technical liaison into this consortium. We're the layer between you and the shared Empire AI infrastructure.
So, how do you as the users of Empire AI get help when you have questions or run into issues?
Short answer, start with us.
On the Columbia side, our… your first stop should be to send a new email to empireai-support at columbia.edu. Emailing that address opens up a ticket automatically in our ServiceNow ticketing queue, so it's tracked and routed properly.
Our whole team gets notified when new tickets are created, and any one of us can assist from there.
Note, we are a small team of four, supporting a growing Columbia user base, so please route requests through this email rather than messaging anyone individually. That way, nothing falls through the cracks when someone's out.
And to break it down, for anything about your account, your project, your allocation, billing, that's us, frontline local support.
Empire AI's team maintains the data center operations, hardware maintenance, backend infrastructure for managed platforms.
And you can always email them via help at empireai.edu, but note that it gets routed to their managed ticketing system first, which is overseen by their team, and then reassigned to us, so there can be delays in response time.
Emailing our support email gets you the fastest response, even for questions that are technically about Empire AI's platform, because we triage first, and if it turns out to be an infrastructure issue on their end, we can escalate it to the right exact person on the Empire AI team, ourselves, and can follow up with them directly.
You don't need to figure out which bucket your question falls into before you ask.
Some other helpful resources. We also have the Columbia Empire AI Community Slack. The invite's gonna be added to the chat. This is used to communicate general updates, maintenance, or service disruptions, so you'll always know what's going on with the clusters.
It's also a great place to connect with other Columbia researchers using Empire AI, swap tips, best practices.
There's also an Empire AI status page, so you can get real-time updates on the current health of the systems.
We also have a Confluence knowledge base with detailed documentation on navigating the Empire AI resources that we are currently building out.
Empire AI does have their own knowledge base as well, but it can feel a bit overwhelming, so we'll be consolidating all of their information into our knowledge base for ease of reference.
So, to summarize, start with emailing us, we'll triage, address, and route it from there.
So, now let's just quickly zoom out for a second, especially for anyone newer to Empire AI on today's call.
As mentioned, Empire AI is a shared multi-institutional AI compute consortium. It's a pool of large-scale GPU and CPU resources that Columbia researchers can access alongside other researchers at other institutions.
The mission behind it is to give New York's academic researchers the advanced computing they need to do AI research for good. And as a reminder, only individuals holding a full-time PI-eligible appointment at Columbia can submit project proposals for Empire AI.
And, you know, to be clear, Empire AI isn't the only compute resource Columbia offers. We also have our on-prem HPC clusters, a secure data enclave for sensitive data workloads, the Columbia Data Platform, and a broader range of research software in our catalog. So, if you have any questions on other resources, our team is happy to discuss with you further.
Now, let's talk hardware.
There are 3 compute resources on Empire AI, Alpha, Grace, and Beta.
Alpha is the H100 and H200 GPU pool.
18 nodes of 8 H180GB GPUs each, plus 6 nodes of 8 H200s, 141 gigabytes, GPUs each.
For a total of 192 GPUs total. So this is the everyday workhorse for GPU training, fine-tuning, interactive debugging. If you're not sure where to start, alpha is almost always the right answer.
Grace is different in kind. It's CPU only, ARM64 architecture, 60 nodes with 144 CPUs each.
Grace is where you'll go for preprocessing, NPI simulations, CPU-heavy pipelines, things that don't need a GPU, or where you're preparing data before a GPU job.
Beta is Empire AI's newest and largest system, a GB200 and VL72 SuperPod.
with 288 B200 GPUs across 4 racks.
This is Blackwell generation hardware, built for the largest training and inference workloads at scale. So, multi-node jobs that genuinely need that scale.
So, given those three options, how do you actually decide
This will largely depend on your workload needs. So, if you need established H100 training, or you're doing interactive GPU debugging, I'd say go with the Alpha H100s.
If your model or batch size needs more GPU memory headroom than the H100's 80GB gives you, stay on Alpha, but request maybe the H200 nodes instead. So, same system, more memory per GPU.
If your workload is CPU heavy, so preprocessing, maybe ARM64 validation needed, MPI-based simulation, use Grace. You're not gonna get the GPU acceleration there, but you'll get a lot more CPU parallelism per job.
And if you need the newest Blackwell-era hardware, or you're running genuinely large multi-GPU jobs at scale, go with Beta. So, while Beta is the newest and highest demand system, users will still need to size their jobs appropriately, rather than defaulting there for everything.
Alpha will probably cover the vast majority of day-to-day work.
Now, storage and data.
So, home directories, each user is provided a home directory at the following path, and the scratch system is also mounted at the following path listed here on the slide.
Regarding storage allocations, every user gets 100GB of free home directory storage automatically.
Project storage can be up to 100 terabytes, and this is covered under the standard service unit billing. We'll go more into service units in a second.
If your project needs more than 100 terabytes.
That will need to be reviewed and approved on a case-by-case basis, so reach out to us if you think you'll need it. And, you know, nope, storage is not infinite on Empire AI.
For moving data in and out of Empire AI, you can use Globus.
To use it, you'll need to make sure you have three things. Globus account, access to Columbia's authentication system, and the Active Empire AI account.
So, the steps are fairly straightforward. You first go to the Globalist File Manager at the following link that will be provided, log in with your Columbia credentials, search Empire AI in the Collections search bar, and select it.
Click Open in File Manager, and then from there, you can transfer your files.
The first time you do this, Globus will prompt you to authenticate with your Empire AI account and complete a consent step, but it should be a one-time thing during initial setup, not something you'll have to do or repeat on every transfer.
Regarding data classification and compliance.
Empire AI is not currently certified to hold or process sensitive data. That includes PII, RHI, and PHI, so if your research involves any of those categories, you'll need to de-identify and anonymize the data before it goes on Empire AI systems.
In the future, Empire AI plans to handle sensitive data for a subset of their system, and will communicate updates as we hear from their team.
If you're not sure how your data is classified, you can refer to Columbia's data classification policy, and if you're unsure about that, you know, after reading it, you can feel free to talk to us.
Secondly, if your collaborators or lab members work internationally, Empire AI is a export-controlled system, so under U.S. Department of Commerce export administration regulations.
Remote access to Empire AI is prohibited while a user is physically located in any of the restricted country groups.
So, if you have collaborators who are traveling internationally, or based abroad, or you're traveling, this is worth flagging directly, as access can be blocked based on physical location at the time of the connection attempt.
Alright, so let's talk filling, because this is changing soon, and I want everyone to leave today with the most updated information.
Service units, or SUs, are the currency used to charge compute usage on Empire AI. It follows the same model that the National Science Foundation uses for its access allocation program, so if you're familiar or have used access resources before, this will feel a bit familiar.
Compute usage on Empire AI is free now through September 30th, 2026, and service unit billing will start October 1st, 2026.
And once this billing starts, the rate for Columbia projects is 25 cents per SU.
Empire AI's standard rate for all the other institutions is 50 cents per SU, but Columbia is subsidizing 50% of compute costs for our researchers.
On alpha, 1 GPU hour equals 1 SU. On Grace, 1 node hour equals half a SU. And on Beta, 1 GPU hour equals 2 SUs.
So, the general billing formula is SU cost equals your usage times hours times the rate for that specific resource you're requesting.
So, if you're planning a large training run on beta, it's worth doing that math ahead of time so that there's no surprises on your allocation balance after October 1st. And we highly recommend utilizing the resources now while they're still available at no charge, and you can get a sense of your SU costs going forward.
Before billing kicks in on October 1st, PIs will need to purchase service units in advance.
Any projects that have not purchased any service units by September 22nd will have their projects set to zero service units, and their access restricted to Empire AI resources. So, the question is, how do you purchase service units?
You can use the link provided that we'll be posting in the Zoom chat, and you'll be going to the iLab portal. If you're a sponsored PI, your account should be mapped automatically already in iLab.
On the Services page, you'll expand the Empire AI Service drop-down, and click Request Service.
You'll enter the number of SUs you need, and as shared in previous communications, initial requests are capped at 2,000 SUs to ensure fair usage of the resources, so anything larger will be sent back from our team for revision.
If you run out of SUs, PIs will have the ability to request additional top-ups throughout the year.
As you're going through the process, in the payment information section, you'll select the charge string or charge strings that will fund the request, and you can even split requests across multiple charge strings by percentage, and then you'll submit the request to finalize the order.
And then wait for our confirmation. So, you'll get a confirmation email right away from iLab, and a second notice, once it's processed by our team, will then process your request, and the SU credits will update in your account within 2-3 business days.
If you do run into any issues with iLab itself, or need to update your chart string information, you can always email the iLab email provided with your PI's name, email, the service requested, and any other relevant details, and they'll help sort it out.
So, quick overview of the steps that I've just mentioned. The link here will redirect you to the following page on the left. And here, you'll click on Empire AI. Moving over to the right side, you'll see the Empire AI service. You'll click Request Service.
Enter your desired service units in the quantity field, select your charge string, and then submit your request.
We will be sending out individual reminders to PIs that we have not yet received iLab service unit requests from, so we'll be doing that in the coming weeks. So, if you have already submitted a request, you know, not… you don't have to worry, we'll be processing them starting September 22nd, ahead of the October 1st billing date.
So, that's how PIs acquire SUs. Now, let's look at the programs that power where you'll actually manage your accounts and projects on the day-to-day, starting with account creation and authentication that's powered by Fido. So…
Account creation, password resets, two-factor authentication enrollment can all be done on this platform called Fido. This is where we as Columbia Technical Points of Contact can actually help with resetting accounts, passwords, and also help create new Empire AI accounts.
So, one thing we did want to highlight here is that PIs can also now provision accounts for their users directly through FIDO. To do that, you'll need the person's full name, email address, and their phone number.
We'll go through the steps on how an account is created in a second, but, you know, for your convenience, feel free to always just send an email to our team at empireai-support, and we will help with provisioning accounts on your behalf.
So, while Fido manages your individual account credentials, another platform, Coldfront manages your project. So, this is membership to your project, access, and your project SU usage.
Here, technical points of contact, like ourselves and our team and PIs, can add and remove users from their project. They can review their service unit usage.
And something that people often miss is this is where you're going to actually find your Slurm project account name. So, that project name is what you'll actually plug into your job submission scripts. So, if a job submission is failing because Slurm doesn't recognize your account, you can double-check it here.
You can also see on the screenshot on the left what a project page generally looks like. It'll have the PI information, project status,field of science, and below that, we'll list, then, the list of project members with their roles, and then on the left here, we have a screenshot of allocation usage with a bar at the bottom that shows total SUs used on your project so far.
And, you know, if you ever suspect something's wrong on the platform side, like a job that should be running isn't, or you're getting unexpected errors, you can check Empire AI's status page. So here's where you'll get real-time status and incident history across Alpha, beta, Grace, the storage.
So, I highly recommend bookmarking and subscribing to the updates on this page, so you'll get notified in real time if anything is ever going on.
Now, going through what the onboarding process actually looks like for those new to Empire AI, or for PIs that are bringing on new members at this time.
In the scenario I have listed here, you've already reached out to EmpireAI-Support about a new account request. For example, a new PhD student on your project, and you've provided all of their info.
to get started on provisioning their account. So, they'll get an email from Fido at EmpireAI.edu with their account details. We'll then send them the activation PIN via text message to the individual, never by email for security reasons.
With setting up their password, they'll then click on the link in the email and enter their PIN when prompted.
And then the final step is to enroll in two-factor authentication. They're gonna SSH into Alpha, enter their new credentials, and then they'll be prompted to scan a QR code for setting up an authenticator app, which you can do with either Duo or Google Authenticator to link their account.
So, once that's linked, the Authenticator app is what generates the six-digit codes that you'll be prompted for when you log into Empire AI.
So, again, PIs, you can create accounts for your own users directly in Fido. You'll just have to, you know, get all of that information that I just mentioned, and for your convenience, you know, our team can provision accounts on your behalf, as long as we have PI approval.
Once your account is active, here's how you actually get in. So, for alpha, ssh to alpha.empire-ai.org.
Grace also shares the same login as Alpha, so there's no separate Grace hostname. So, if you want to use the Grace CP resources, you'll just have to SSH into Alpha.
For beta, SSH to beta.empireai.edu, so note the different host names for using beta resources.
In either path, you're going to be prompted for the password, followed by the 6-digit authenticator code that was set up.
Now, for all these three resources, Alpha, Grace, Beta, jobs are scheduled by Slurm. This is the workload manager that decides which jobs run when and on what hardware. So, every single job you submit runs under a QoS quality of service tier.
And that tier is what controls three things. Your priority in the queue, your max wall time, and your service unit billing rate.
So, there are 5 tiers on Empire AI. Test is for quick script validation. It's the cheapest option at half the standard SU rate, you know, meant for making sure your code actually runs before you commit real resources to it.
Interactive gives you that live GPU shell for debugging at the standard one times rate.
Standard is the default production tier, running from 1 to 2X, depending on the system and resource requested.
Long is the budget-friendly option for lower priority, longer-running work at half price, so here you're trading priority for SU cost savings.
And priority for any kind of
Deadline-driven work, urgent work, running from 2X to 4X.
Depending on which resource you requested, so you're paying more, but you get to jump the queue.
you know, exact wall time limits and rates vary slightly system to system, so the specific numbers for any given tier, you can check on our Empire AI knowledge base on those numbers.
Now, diving a bit deeper and looking at some sample interactive jobs on the three systems.
On each system, you'll need to specify the partition hardware. So, for alpha, dash dash partition equals alpha, and here's a sample interactive session and what it'll look like.
So, in the scenario we have for Alpha here, this requests one GPU under the interactive QoS tier for 1 hour, and drops you into a live shell.
on Grace, the partition is Grace, and remember, you're still logging into that same Alpha host, since Grace shares this alpha login node.
the command looks similar in structure, but different in terms of the resource flags, because Grace is CPU only, so…
For example, here, note that there's no dash dash gras equals GPU flag here, you're requesting CPU-only resources, and instead, in this example, you're specifying one full node and 144 tasks per node, matching Grace per node CPU count.
On Beta, the partition is beta, and notice the dash dash account, your account flag. Beta requires you to explicitly specify your Slurm account project name, which, as I mentioned earlier, you can find in ColdFront under your project allocation details.
If you leave that flag off on beta, your job will likely fail or get rejected at submission. So, key details to remember here are to know your partition.
You know, make sure you list the number of resources and your project name.
putting this whole workflow together, I also provide an example of what a multi-GPU training job would look like on Beta. So, the typical workflow here has four steps. You know, first, you're writing your JavaScript, so a shell script with S-batch directives.
at the top, specifying your partition, the account, QoS tier, GPU count, wall time. You submit it, so either SBatch, you know, script.sh to run in the background as a batch job, or with SRUN, if you want a live interactive shell instead.
Monitor your progress while using SQ while it runs, and once it's done, you can check your results, you know, review the output files, error log files, and if the job was re-queued at any point, you know, verify your checkpoints that came through cleanly.
So here's what it looks like as a script on the right for a 4GPU training job on beta. And, you know, you would submit this with the simple spatch train.sh.
So, some key things I did include in the script. For example, the dash dash container image flag.
Here, we're pulling in the NVIDIA NGC container, PyTorch in this case, so your environment is fully specified and reproducible.
If you aren't familiar with what containers are, they help make your application portable, so it's an additional tool to produce a reproducible environment, so you're not relying on whatever happens to be installed or not installed on the node.
Other two lines that are above work together. This is important for any job that might be running longer than a couple of hours. The dash dash re-queue tells Slurm it's allowed to automatically resubmit your job if it gets preempted or interrupted.
And the dash dash signal line sends your script a warning signal 900 seconds, or 15 minutes before your wall time limit hits. So, this gives your training script a window to checkpoint its current state and exit cleanly, rather than getting killed mid-step and losing progress.
So, general rule to remember, you know, no QS…
tier grads unlimited wall time. Every tier has a wall time cap, so for any job that might run longer than that cap, which for long training runs is basically guaranteed, pairing that re-queue with your own checkpoint logic is how you can avoid losing hours of compute to a wall time cutoff.
And some other helpful tips with monitoring and managing jobs.
To see what's currently running or queued under your name, you can use the SQ command here for job history, including status and resources used. You can use the S account.
If you're ever unsure which accounts and QoS tiers you personally have access to, you can run the following command, the account manager command, and they'll give you, you know, save you time from doing a guessing game of which, you know, account value to use.
If for any reason you don't see the Slurm project account in… when you run this, you know, there might be something on our end or in Cold Front that we will have to do to manually add you to the project, then we can always do that.
to check your fair share standing, which affects how quickly your jobs get scheduled relative to other users. You know, you can use the SSHARE here, and if you need to kill a job that's running or still pending in the queue, you know, use SCANCEL followed by the job ID.
So, I genuinely recommend, you know, just keeping these five commands somewhere handy if you're not already familiar, because they cover a majority of what you'll need day-to-day once you're actively running and troubleshooting jobs.
And a quick funding announcement.
This was forwarded to us on behalf of the Simons Foundation for Empire AI.
Utilizing the full power of AI is a $6 million total program budget allocation with 6 to 12 awardees anticipated, and the deadline for this funding opportunity is December 1st at 12 p.m. Eastern Time.
So, the focus areas are computational astrophysics, biology, math, neuroscience, physical chemistry, you know, proposals that are developing foundational AI ML models, addressing genuinely fundamental scientific questions, and
They must… the proposals must, you know, explicitly detail the use of multi-node GPU clusters through Empire AI. On, you know, topic of eligibility, the lead PI needs to be either tenure track or tenured faculty at a consortium institution, including Columbia.
And all key personnel will need to hold either a PhD, MD, or equivalent. And this award period is going to run from June 1st, 2027 through May 31st, 2029.
Applications for this, you know, funding opportunity go through the Simons Award Manager or SAM platform that they have, and we're gonna provide the link in the chat if you have not seen the email communications that we've shared previously.
If you have any questions about the funding opportunity, you can reach out to the following principal gifts officer or executive director of UCFR. Both contacts are available on this slide.
And, you know, before we start opening it up, I want to share what we've heard so far from the Columbia community. You know, we really value everyone's feedback and recognize that there were a lot of bumps along the way to get the systems and support to where they are now.
Regarding resource usage patterns so far. For those that have been actively using Empire AI, we've seen that over 60% have used Alpha, nearly 30% have used Beta, and nearly 10% Grace. But we also did see that about 36% of those surveyed have yet to use any Empire AI resources, which tells us there's real room to grow adoption here.
on confidence and satisfaction levels, the average confidence navigating Empire AI came at 2.82 out of 5, and average satisfaction at 2.91 out of 5. So, you know, these numbers tell us we still have work to do, and we're taking that feedback seriously.
The top challenges that individuals flagged were resource availability and wall times, along with limited exposure to the platform so far. And, you know, just to provide some, you know, background context.
Over the past few months, the Empire AI team have been working to stabilize the beta clusters, and have also resolved the faulty leak detection alarms that were going off, which attributed to a lot of these issues. So, you know, we really appreciate everyone's patience during this time, and
going forward, users should see improved wait times and stable access to the system. So, you know, leveraging the QoS tiers as much as possible, such as the priority tier, should also help.
And the improvements that, you know, people were asking for are also really clear. Better documentation, better usability, which is exactly what our team has been actively working on since we've received these feedback.
We are building out that Confluence knowledge base with detailed documentation on Empire AI, and we'll use future user sessions to work with you and your research groups on navigating Empire AI and triaging issues as they come up.
So, if you haven't taken the survey yet, please do. I've copied the link here, and a colleague is gonna put it also in the Zoom chat.
Your feedback directly shapes what we prioritize next, and we would love to hear from the user community about where we should be going, are there things we should be improving, what's going well, what's not?
And thank you for sticking with me through the entire presentation!
And now, I'd like to, you know, finally open it up to everyone on this call, whether it's about getting access, the billing charge coming up October 1st, you know, job submissions.
Anything else we covered or didn't cover, feel free to ask, and if a question comes up after today, or you think of something later on, like, feel free to always email us at empireai-support at columbia.edu.
Also, if anyone has any suggestions or requests for what they would like future sessions to look like, please let us know, and we'll do our best to accommodate.
For example, you know, if people are interested in more in-depth material on navigating Empire AI, or we can also collaborate with the NVIDIA team that's supporting Empire AI to maybe potentially host a feature user session, diving deeper into the resources that they provide, you know, we're open to all of this feedback and more.
This session is the first of a series that we've started for the Columbia Empire AI research community. Some general housekeeping before we begin. If you have a question throughout the presentation, feel free to drop it in the chat, and someone on our team will get to it, or hold it for the Q&A block at the end.
We'll also be recording today's session, so please meet yourselves and turn off your video, and a recording will be made available on our Research Computing Services video library later on.
We'll also share out the slides and the follow-up email communication. So, with that out of the way, let's get into it.
Looking at today's agenda, we'll cover who we are, what our team supports, a quick overview of Empire AI, walk through how to navigate the Empire AI resources, share a new funding opportunity, review feedback we have received so far from the user community, and leave room at the end for open Q&A.
Who we have here are the Columbia Technical Points of Contact. We are the team you'll interact with most directly for anything Empire AI-related on the Columbia side. Alongside me, we have Al, Waqas, and Halyna, our Senior Research Systems Engineers who handle day-to-day technical support for Empire AI, in addition to our on-prem HPC clusters.
The Columbia Technical Points of Contact sit within the Research Computing Services Department at CUIT, and we actively collaborate with the Office of the Executive Vice President for Research.
Empire AI itself is a separate multi-institutional consortium. We're one of eight member institutions alongside Cornell, NYU, the CUNYs, the SUNYs, and other institutions. Each institution also has their own team of institutional technical points of contact.
And our team is Columbia's technical liaison into this consortium. We're the layer between you and the shared Empire AI infrastructure.
So, how do you as the users of Empire AI get help when you have questions or run into issues?
Short answer, start with us.
On the Columbia side, our… your first stop should be to send a new email to empireai-support at columbia.edu. Emailing that address opens up a ticket automatically in our ServiceNow ticketing queue, so it's tracked and routed properly.
Our whole team gets notified when new tickets are created, and any one of us can assist from there.
Note, we are a small team of four, supporting a growing Columbia user base, so please route requests through this email rather than messaging anyone individually. That way, nothing falls through the cracks when someone's out.
And to break it down, for anything about your account, your project, your allocation, billing, that's us, frontline local support.
Empire AI's team maintains the data center operations, hardware maintenance, backend infrastructure for managed platforms.
And you can always email them via help at empireai.edu, but note that it gets routed to their managed ticketing system first, which is overseen by their team, and then reassigned to us, so there can be delays in response time.
Emailing our support email gets you the fastest response, even for questions that are technically about Empire AI's platform, because we triage first, and if it turns out to be an infrastructure issue on their end, we can escalate it to the right exact person on the Empire AI team, ourselves, and can follow up with them directly.
You don't need to figure out which bucket your question falls into before you ask.
Some other helpful resources. We also have the Columbia Empire AI Community Slack. The invite's gonna be added to the chat. This is used to communicate general updates, maintenance, or service disruptions, so you'll always know what's going on with the clusters.
It's also a great place to connect with other Columbia researchers using Empire AI, swap tips, best practices.
There's also an Empire AI status page, so you can get real-time updates on the current health of the systems.
We also have a Confluence knowledge base with detailed documentation on navigating the Empire AI resources that we are currently building out.
Empire AI does have their own knowledge base as well, but it can feel a bit overwhelming, so we'll be consolidating all of their information into our knowledge base for ease of reference.
So, to summarize, start with emailing us, we'll triage, address, and route it from there.
So, now let's just quickly zoom out for a second, especially for anyone newer to Empire AI on today's call.
As mentioned, Empire AI is a shared multi-institutional AI compute consortium. It's a pool of large-scale GPU and CPU resources that Columbia researchers can access alongside other researchers at other institutions.
The mission behind it is to give New York's academic researchers the advanced computing they need to do AI research for good. And as a reminder, only individuals holding a full-time PI-eligible appointment at Columbia can submit project proposals for Empire AI.
And, you know, to be clear, Empire AI isn't the only compute resource Columbia offers. We also have our on-prem HPC clusters, a secure data enclave for sensitive data workloads, the Columbia Data Platform, and a broader range of research software in our catalog. So, if you have any questions on other resources, our team is happy to discuss with you further.
Now, let's talk hardware.
There are 3 compute resources on Empire AI, Alpha, Grace, and Beta.
Alpha is the H100 and H200 GPU pool.
18 nodes of 8 H180GB GPUs each, plus 6 nodes of 8 H200s, 141 gigabytes, GPUs each.
For a total of 192 GPUs total. So this is the everyday workhorse for GPU training, fine-tuning, interactive debugging. If you're not sure where to start, alpha is almost always the right answer.
Grace is different in kind. It's CPU only, ARM64 architecture, 60 nodes with 144 CPUs each.
Grace is where you'll go for preprocessing, NPI simulations, CPU-heavy pipelines, things that don't need a GPU, or where you're preparing data before a GPU job.
Beta is Empire AI's newest and largest system, a GB200 and VL72 SuperPod.
with 288 B200 GPUs across 4 racks.
This is Blackwell generation hardware, built for the largest training and inference workloads at scale. So, multi-node jobs that genuinely need that scale.
So, given those three options, how do you actually decide
This will largely depend on your workload needs. So, if you need established H100 training, or you're doing interactive GPU debugging, I'd say go with the Alpha H100s.
If your model or batch size needs more GPU memory headroom than the H100's 80GB gives you, stay on Alpha, but request maybe the H200 nodes instead. So, same system, more memory per GPU.
If your workload is CPU heavy, so preprocessing, maybe ARM64 validation needed, MPI-based simulation, use Grace. You're not gonna get the GPU acceleration there, but you'll get a lot more CPU parallelism per job.
And if you need the newest Blackwell-era hardware, or you're running genuinely large multi-GPU jobs at scale, go with Beta. So, while Beta is the newest and highest demand system, users will still need to size their jobs appropriately, rather than defaulting there for everything.
Alpha will probably cover the vast majority of day-to-day work.
Now, storage and data.
So, home directories, each user is provided a home directory at the following path, and the scratch system is also mounted at the following path listed here on the slide.
Regarding storage allocations, every user gets 100GB of free home directory storage automatically.
Project storage can be up to 100 terabytes, and this is covered under the standard service unit billing. We'll go more into service units in a second.
If your project needs more than 100 terabytes.
That will need to be reviewed and approved on a case-by-case basis, so reach out to us if you think you'll need it. And, you know, nope, storage is not infinite on Empire AI.
For moving data in and out of Empire AI, you can use Globus.
To use it, you'll need to make sure you have three things. Globus account, access to Columbia's authentication system, and the Active Empire AI account.
So, the steps are fairly straightforward. You first go to the Globalist File Manager at the following link that will be provided, log in with your Columbia credentials, search Empire AI in the Collections search bar, and select it.
Click Open in File Manager, and then from there, you can transfer your files.
The first time you do this, Globus will prompt you to authenticate with your Empire AI account and complete a consent step, but it should be a one-time thing during initial setup, not something you'll have to do or repeat on every transfer.
Regarding data classification and compliance.
Empire AI is not currently certified to hold or process sensitive data. That includes PII, RHI, and PHI, so if your research involves any of those categories, you'll need to de-identify and anonymize the data before it goes on Empire AI systems.
In the future, Empire AI plans to handle sensitive data for a subset of their system, and will communicate updates as we hear from their team.
If you're not sure how your data is classified, you can refer to Columbia's data classification policy, and if you're unsure about that, you know, after reading it, you can feel free to talk to us.
Secondly, if your collaborators or lab members work internationally, Empire AI is a export-controlled system, so under U.S. Department of Commerce export administration regulations.
Remote access to Empire AI is prohibited while a user is physically located in any of the restricted country groups.
So, if you have collaborators who are traveling internationally, or based abroad, or you're traveling, this is worth flagging directly, as access can be blocked based on physical location at the time of the connection attempt.
Alright, so let's talk filling, because this is changing soon, and I want everyone to leave today with the most updated information.
Service units, or SUs, are the currency used to charge compute usage on Empire AI. It follows the same model that the National Science Foundation uses for its access allocation program, so if you're familiar or have used access resources before, this will feel a bit familiar.
Compute usage on Empire AI is free now through September 30th, 2026, and service unit billing will start October 1st, 2026.
And once this billing starts, the rate for Columbia projects is 25 cents per SU.
Empire AI's standard rate for all the other institutions is 50 cents per SU, but Columbia is subsidizing 50% of compute costs for our researchers.
On alpha, 1 GPU hour equals 1 SU. On Grace, 1 node hour equals half a SU. And on Beta, 1 GPU hour equals 2 SUs.
So, the general billing formula is SU cost equals your usage times hours times the rate for that specific resource you're requesting.
So, if you're planning a large training run on beta, it's worth doing that math ahead of time so that there's no surprises on your allocation balance after October 1st. And we highly recommend utilizing the resources now while they're still available at no charge, and you can get a sense of your SU costs going forward.
Before billing kicks in on October 1st, PIs will need to purchase service units in advance.
Any projects that have not purchased any service units by September 22nd will have their projects set to zero service units, and their access restricted to Empire AI resources. So, the question is, how do you purchase service units?
You can use the link provided that we'll be posting in the Zoom chat, and you'll be going to the iLab portal. If you're a sponsored PI, your account should be mapped automatically already in iLab.
On the Services page, you'll expand the Empire AI Service drop-down, and click Request Service.
You'll enter the number of SUs you need, and as shared in previous communications, initial requests are capped at 2,000 SUs to ensure fair usage of the resources, so anything larger will be sent back from our team for revision.
If you run out of SUs, PIs will have the ability to request additional top-ups throughout the year.
As you're going through the process, in the payment information section, you'll select the charge string or charge strings that will fund the request, and you can even split requests across multiple charge strings by percentage, and then you'll submit the request to finalize the order.
And then wait for our confirmation. So, you'll get a confirmation email right away from iLab, and a second notice, once it's processed by our team, will then process your request, and the SU credits will update in your account within 2-3 business days.
If you do run into any issues with iLab itself, or need to update your chart string information, you can always email the iLab email provided with your PI's name, email, the service requested, and any other relevant details, and they'll help sort it out.
So, quick overview of the steps that I've just mentioned. The link here will redirect you to the following page on the left. And here, you'll click on Empire AI. Moving over to the right side, you'll see the Empire AI service. You'll click Request Service.
Enter your desired service units in the quantity field, select your charge string, and then submit your request.
We will be sending out individual reminders to PIs that we have not yet received iLab service unit requests from, so we'll be doing that in the coming weeks. So, if you have already submitted a request, you know, not… you don't have to worry, we'll be processing them starting September 22nd, ahead of the October 1st billing date.
So, that's how PIs acquire SUs. Now, let's look at the programs that power where you'll actually manage your accounts and projects on the day-to-day, starting with account creation and authentication that's powered by Fido. So…
Account creation, password resets, two-factor authentication enrollment can all be done on this platform called Fido. This is where we as Columbia Technical Points of Contact can actually help with resetting accounts, passwords, and also help create new Empire AI accounts.
So, one thing we did want to highlight here is that PIs can also now provision accounts for their users directly through FIDO. To do that, you'll need the person's full name, email address, and their phone number.
We'll go through the steps on how an account is created in a second, but, you know, for your convenience, feel free to always just send an email to our team at empireai-support, and we will help with provisioning accounts on your behalf.
So, while Fido manages your individual account credentials, another platform, Coldfront manages your project. So, this is membership to your project, access, and your project SU usage.
Here, technical points of contact, like ourselves and our team and PIs, can add and remove users from their project. They can review their service unit usage.
And something that people often miss is this is where you're going to actually find your Slurm project account name. So, that project name is what you'll actually plug into your job submission scripts. So, if a job submission is failing because Slurm doesn't recognize your account, you can double-check it here.
You can also see on the screenshot on the left what a project page generally looks like. It'll have the PI information, project status,field of science, and below that, we'll list, then, the list of project members with their roles, and then on the left here, we have a screenshot of allocation usage with a bar at the bottom that shows total SUs used on your project so far.
And, you know, if you ever suspect something's wrong on the platform side, like a job that should be running isn't, or you're getting unexpected errors, you can check Empire AI's status page. So here's where you'll get real-time status and incident history across Alpha, beta, Grace, the storage.
So, I highly recommend bookmarking and subscribing to the updates on this page, so you'll get notified in real time if anything is ever going on.
Now, going through what the onboarding process actually looks like for those new to Empire AI, or for PIs that are bringing on new members at this time.
In the scenario I have listed here, you've already reached out to EmpireAI-Support about a new account request. For example, a new PhD student on your project, and you've provided all of their info.
to get started on provisioning their account. So, they'll get an email from Fido at EmpireAI.edu with their account details. We'll then send them the activation PIN via text message to the individual, never by email for security reasons.
With setting up their password, they'll then click on the link in the email and enter their PIN when prompted.
And then the final step is to enroll in two-factor authentication. They're gonna SSH into Alpha, enter their new credentials, and then they'll be prompted to scan a QR code for setting up an authenticator app, which you can do with either Duo or Google Authenticator to link their account.
So, once that's linked, the Authenticator app is what generates the six-digit codes that you'll be prompted for when you log into Empire AI.
So, again, PIs, you can create accounts for your own users directly in Fido. You'll just have to, you know, get all of that information that I just mentioned, and for your convenience, you know, our team can provision accounts on your behalf, as long as we have PI approval.
Once your account is active, here's how you actually get in. So, for alpha, ssh to alpha.empire-ai.org.
Grace also shares the same login as Alpha, so there's no separate Grace hostname. So, if you want to use the Grace CP resources, you'll just have to SSH into Alpha.
For beta, SSH to beta.empireai.edu, so note the different host names for using beta resources.
In either path, you're going to be prompted for the password, followed by the 6-digit authenticator code that was set up.
Now, for all these three resources, Alpha, Grace, Beta, jobs are scheduled by Slurm. This is the workload manager that decides which jobs run when and on what hardware. So, every single job you submit runs under a QoS quality of service tier.
And that tier is what controls three things. Your priority in the queue, your max wall time, and your service unit billing rate.
So, there are 5 tiers on Empire AI. Test is for quick script validation. It's the cheapest option at half the standard SU rate, you know, meant for making sure your code actually runs before you commit real resources to it.
Interactive gives you that live GPU shell for debugging at the standard one times rate.
Standard is the default production tier, running from 1 to 2X, depending on the system and resource requested.
Long is the budget-friendly option for lower priority, longer-running work at half price, so here you're trading priority for SU cost savings.
And priority for any kind of
Deadline-driven work, urgent work, running from 2X to 4X.
Depending on which resource you requested, so you're paying more, but you get to jump the queue.
you know, exact wall time limits and rates vary slightly system to system, so the specific numbers for any given tier, you can check on our Empire AI knowledge base on those numbers.
Now, diving a bit deeper and looking at some sample interactive jobs on the three systems.
On each system, you'll need to specify the partition hardware. So, for alpha, dash dash partition equals alpha, and here's a sample interactive session and what it'll look like.
So, in the scenario we have for Alpha here, this requests one GPU under the interactive QoS tier for 1 hour, and drops you into a live shell.
on Grace, the partition is Grace, and remember, you're still logging into that same Alpha host, since Grace shares this alpha login node.
the command looks similar in structure, but different in terms of the resource flags, because Grace is CPU only, so…
For example, here, note that there's no dash dash gras equals GPU flag here, you're requesting CPU-only resources, and instead, in this example, you're specifying one full node and 144 tasks per node, matching Grace per node CPU count.
On Beta, the partition is beta, and notice the dash dash account, your account flag. Beta requires you to explicitly specify your Slurm account project name, which, as I mentioned earlier, you can find in ColdFront under your project allocation details.
If you leave that flag off on beta, your job will likely fail or get rejected at submission. So, key details to remember here are to know your partition.
You know, make sure you list the number of resources and your project name.
putting this whole workflow together, I also provide an example of what a multi-GPU training job would look like on Beta. So, the typical workflow here has four steps. You know, first, you're writing your JavaScript, so a shell script with S-batch directives.
at the top, specifying your partition, the account, QoS tier, GPU count, wall time. You submit it, so either SBatch, you know, script.sh to run in the background as a batch job, or with SRUN, if you want a live interactive shell instead.
Monitor your progress while using SQ while it runs, and once it's done, you can check your results, you know, review the output files, error log files, and if the job was re-queued at any point, you know, verify your checkpoints that came through cleanly.
So here's what it looks like as a script on the right for a 4GPU training job on beta. And, you know, you would submit this with the simple spatch train.sh.
So, some key things I did include in the script. For example, the dash dash container image flag.
Here, we're pulling in the NVIDIA NGC container, PyTorch in this case, so your environment is fully specified and reproducible.
If you aren't familiar with what containers are, they help make your application portable, so it's an additional tool to produce a reproducible environment, so you're not relying on whatever happens to be installed or not installed on the node.
Other two lines that are above work together. This is important for any job that might be running longer than a couple of hours. The dash dash re-queue tells Slurm it's allowed to automatically resubmit your job if it gets preempted or interrupted.
And the dash dash signal line sends your script a warning signal 900 seconds, or 15 minutes before your wall time limit hits. So, this gives your training script a window to checkpoint its current state and exit cleanly, rather than getting killed mid-step and losing progress.
So, general rule to remember, you know, no QS…
tier grads unlimited wall time. Every tier has a wall time cap, so for any job that might run longer than that cap, which for long training runs is basically guaranteed, pairing that re-queue with your own checkpoint logic is how you can avoid losing hours of compute to a wall time cutoff.
And some other helpful tips with monitoring and managing jobs.
To see what's currently running or queued under your name, you can use the SQ command here for job history, including status and resources used. You can use the S account.
If you're ever unsure which accounts and QoS tiers you personally have access to, you can run the following command, the account manager command, and they'll give you, you know, save you time from doing a guessing game of which, you know, account value to use.
If for any reason you don't see the Slurm project account in… when you run this, you know, there might be something on our end or in Cold Front that we will have to do to manually add you to the project, then we can always do that.
to check your fair share standing, which affects how quickly your jobs get scheduled relative to other users. You know, you can use the SSHARE here, and if you need to kill a job that's running or still pending in the queue, you know, use SCANCEL followed by the job ID.
So, I genuinely recommend, you know, just keeping these five commands somewhere handy if you're not already familiar, because they cover a majority of what you'll need day-to-day once you're actively running and troubleshooting jobs.
And a quick funding announcement.
This was forwarded to us on behalf of the Simons Foundation for Empire AI.
Utilizing the full power of AI is a $6 million total program budget allocation with 6 to 12 awardees anticipated, and the deadline for this funding opportunity is December 1st at 12 p.m. Eastern Time.
So, the focus areas are computational astrophysics, biology, math, neuroscience, physical chemistry, you know, proposals that are developing foundational AI ML models, addressing genuinely fundamental scientific questions, and
They must… the proposals must, you know, explicitly detail the use of multi-node GPU clusters through Empire AI. On, you know, topic of eligibility, the lead PI needs to be either tenure track or tenured faculty at a consortium institution, including Columbia.
And all key personnel will need to hold either a PhD, MD, or equivalent. And this award period is going to run from June 1st, 2027 through May 31st, 2029.
Applications for this, you know, funding opportunity go through the Simons Award Manager or SAM platform that they have, and we're gonna provide the link in the chat if you have not seen the email communications that we've shared previously.
If you have any questions about the funding opportunity, you can reach out to the following principal gifts officer or executive director of UCFR. Both contacts are available on this slide.
And, you know, before we start opening it up, I want to share what we've heard so far from the Columbia community. You know, we really value everyone's feedback and recognize that there were a lot of bumps along the way to get the systems and support to where they are now.
Regarding resource usage patterns so far. For those that have been actively using Empire AI, we've seen that over 60% have used Alpha, nearly 30% have used Beta, and nearly 10% Grace. But we also did see that about 36% of those surveyed have yet to use any Empire AI resources, which tells us there's real room to grow adoption here.
on confidence and satisfaction levels, the average confidence navigating Empire AI came at 2.82 out of 5, and average satisfaction at 2.91 out of 5. So, you know, these numbers tell us we still have work to do, and we're taking that feedback seriously.
The top challenges that individuals flagged were resource availability and wall times, along with limited exposure to the platform so far. And, you know, just to provide some, you know, background context.
Over the past few months, the Empire AI team have been working to stabilize the beta clusters, and have also resolved the faulty leak detection alarms that were going off, which attributed to a lot of these issues. So, you know, we really appreciate everyone's patience during this time, and
going forward, users should see improved wait times and stable access to the system. So, you know, leveraging the QoS tiers as much as possible, such as the priority tier, should also help.
And the improvements that, you know, people were asking for are also really clear. Better documentation, better usability, which is exactly what our team has been actively working on since we've received these feedback.
We are building out that Confluence knowledge base with detailed documentation on Empire AI, and we'll use future user sessions to work with you and your research groups on navigating Empire AI and triaging issues as they come up.
So, if you haven't taken the survey yet, please do. I've copied the link here, and a colleague is gonna put it also in the Zoom chat.
Your feedback directly shapes what we prioritize next, and we would love to hear from the user community about where we should be going, are there things we should be improving, what's going well, what's not?
And thank you for sticking with me through the entire presentation!
And now, I'd like to, you know, finally open it up to everyone on this call, whether it's about getting access, the billing charge coming up October 1st, you know, job submissions.
Anything else we covered or didn't cover, feel free to ask, and if a question comes up after today, or you think of something later on, like, feel free to always email us at empireai-support at columbia.edu.
Also, if anyone has any suggestions or requests for what they would like future sessions to look like, please let us know, and we'll do our best to accommodate.
For example, you know, if people are interested in more in-depth material on navigating Empire AI, or we can also collaborate with the NVIDIA team that's supporting Empire AI to maybe potentially host a feature user session, diving deeper into the resources that they provide, you know, we're open to all of this feedback and more.