WEBVTT 00:00:21.000 --> 00:00:22.000 Okay, sorry. Okay, so all good. 00:00:22.000 --> 00:00:28.000 Material. 00:00:28.000 --> 00:00:29.000 Perfect. 00:00:29.000 --> 00:00:42.000 Okay, amazing. Amazing. So thanks for this workshop. I mean, I… didn't manage to attend last two days. It was very late for us. But yeah, I learned a lot from the first two dogs so that's great. 00:00:42.000 --> 00:00:50.000 So now we travel from France to South Africa. Survey a lot of carbon, I guess. 00:00:50.000 --> 00:01:07.000 And yeah, I mean, South Africa, we are so far away, we cannot use trends to go anywhere. We have to fly, unfortunately. But anyway, so yeah, we'll also move from more exciting biodiversity to more mundane computing stuff. 00:01:07.000 --> 00:01:21.000 Uh yeah now I'm no expert. I mean, this is in fact the first time I'm presenting in a meeting like this. So if I use any term Not in the way… more insiders use advert apologies. 00:01:21.000 --> 00:01:37.000 Now for me and i feel sustainability means different things to different people. I mean, of course, there are common broad themes but then more fine-grained meaning is probably different. In this talk, I'm just going to focus on computing aspects. 00:01:37.000 --> 00:01:58.000 More specifically on our sort of frequent colleagues and students we made a survey circulated among the community we got 15 responses not as many as I wanted. The community is probably twice as large as But also, I think it's a fair representation of the community in the sense that we have 00:01:58.000 --> 00:02:09.000 A lot of master's students PhD students and then so yeah as you can see on the right i mean it's It's kind of we we got a fair representation from the community, so that's good. 00:02:09.000 --> 00:02:15.000 Also, I mean. We might think computing does not depend on the country. 00:02:15.000 --> 00:02:23.000 It's not. I mean, I was a postdoc in Europe for many years And whether that's the best way of doing things or not. 00:02:23.000 --> 00:02:29.000 I would just SSH intox plus dox-11 forwarding. 00:02:29.000 --> 00:02:44.000 Look at a plot directly and edit things and look at a plot directly. Again, when I moved to South Africa, I realized that doesn't work. Excellent forwarding does not work. The latency is too much. The upload speed is slow. 00:02:44.000 --> 00:02:56.000 I had to install root. Locally, all my students, all my colleagues I know do that. Again, not a big deal, but just to say like how your workflow changes depending on where you are. 00:02:56.000 --> 00:03:03.000 Now, we started by asking them what are the biggest problems you face? And I kind of alluded to that. 00:03:03.000 --> 00:03:07.000 It's essentially low connection speed. 00:03:07.000 --> 00:03:26.000 And that affects in many ways if you're running stuff on lx plus and the connection gets like we drop the connection and then you have to rerun the thing again Which is not great as any of us who is working on this thing should know 00:03:26.000 --> 00:03:42.000 Also, I mean. We don't really have a lot of local Clusters and grids i mean i know i get i'll get a lot of flack from for saying that but I mean, the ones we have are incredibly hard to use or get access to 00:03:42.000 --> 00:04:03.000 And stuff like that. Yeah, so that's probably that. Now, I was like, why that's the case? I mean, internet is internet And I did not know this before I moved to South Africa. So maybe all of you know this and this is completely redundant, but I found that it is interesting that our internet essentially is 00:04:03.000 --> 00:04:20.000 By undersea cables. Quite a few of them, but they are just cabbles big bad kibbles. And whenever a ship or some like rams into them and just like breaks the cable and that apparently happens like more frequently than we know. 00:04:20.000 --> 00:04:35.000 Our internet is bad. I mean, there are some times when like Countrywide internet is bad for a few days because one of those cables snapped. I mean, this is something we didn't realize before moving to this country and trying to do science. So that's 00:04:35.000 --> 00:04:49.000 Again, probably everyone knows this, but I don't. Okay, so now i'm going to kind divide the topic in three subtopics in the sense like what are the good computing practices? 00:04:49.000 --> 00:05:05.000 So first, of course, is awareness. Do people know i mean all the things we should do. And then are they using that knowledge so that i'll combine those two and then we'll go into how can we address that was the way forward. 00:05:05.000 --> 00:05:22.000 So I know this is slightly Atlas LHC specific, but Big Panda is essentially our grid monitoring thing. So we have computing grid, like most of us know And Atlas has done something very interesting And I don't say super recent. I think this is a few years old 00:05:22.000 --> 00:05:32.000 That when you run a great job, it shows your carbon footprint. I mean, of course, as I learned, it's not the exact amount. It's an average amount depending on CPU consumption and whatnot. 00:05:32.000 --> 00:05:52.000 But it's a number that shows up add the monitoring pitch and 73% of people who responded to the survey said they actually are aware of it. So that's good. At least they know that there is a carbon footprint in running these great jobs, which is relevant for other things because 00:05:52.000 --> 00:06:02.000 If we can run these jobs more optimized way. And then it kind of fits into the fact that Do you know who to ask when you get stuck? 00:06:02.000 --> 00:06:08.000 Because if you don't, then sometimes you just keep running the same failed job again and again, which is not optimal. 00:06:08.000 --> 00:06:15.000 For anything, not for your size, not for your time management, not for carbon footprint, right? So again. 00:06:15.000 --> 00:06:27.000 A large fraction said yes. Probably the same fraction. So that was good. That made me very happy that whether they reach out to them is I don't know, but at least they know. 00:06:27.000 --> 00:06:35.000 Also, Atlas great, and I think this is true for most of the collaborations, have a feature where when you submit a job. 00:06:35.000 --> 00:06:45.000 A small fraction of jobs run and try to see if the jobs would succeed in terms of memory usage in terms of like finding resources and whatnot. 00:06:45.000 --> 00:06:50.000 And if those jobs fail, then the rest of the jobs don't run, which is a good thing, which is again. 00:06:50.000 --> 00:07:05.000 Like a good computing practice and again A decent fraction, like two thirds essentially said they know that that It is a thing. So again, I think that was uh definitely a positive. 00:07:05.000 --> 00:07:11.000 Not only ask them something. Just to check like what what we do. 00:07:11.000 --> 00:07:25.000 And I think as at least as experimentalists, we all have been there you submit a large number of great jobs before going to bed And next morning wake up, everything is red. Everything is a disaster. 00:07:25.000 --> 00:07:33.000 And you cry. And then I think the point is to actually try to figure out why they all failed or you just like. 00:07:33.000 --> 00:07:45.000 No, it must be something else and you resubmit everything again um again half of more than half essentially said they actually try to debug rather than blindly submitting. 00:07:45.000 --> 00:07:56.000 But some of them also said they don't. So I think we need to reach out to everyone and see if we can optimize that. That's part of the training. 00:07:56.000 --> 00:08:06.000 And this is something I learned actually probably last year, that you can visualize contents of a root file. So that means you don't have to download the root file. 00:08:06.000 --> 00:08:19.000 Locally open it and say it and then i mean sometimes you need to do that for your analysis sometimes you just want to check quickly if it's empty or a specific branch exist or something real quickly. You don't need to do all of that. 00:08:19.000 --> 00:08:23.000 And then two-thirds said yes. So all of this is great. 00:08:23.000 --> 00:08:30.000 Then we asked them, have you run Jobs in LX Plus, which is a login node where it should not be running long jobs? 00:08:30.000 --> 00:08:47.000 And then I think 40% said yes which is a bit concerning and that ties to the connectivity issue, right? Because if you don't do that, it's probably like your lxpress connection would probably drop unless you're very, very lucky and then 00:08:47.000 --> 00:08:55.000 You are trying to rerun everything again if it's a long job, you're just wasting a lot of resources So that's not great. 00:08:55.000 --> 00:09:03.000 So what's my overall takeaway from this bird? Considering that we have a significant fraction of younger students. 00:09:03.000 --> 00:09:14.000 I think it's encouraging that they're mostly aware of these issues I also think, because I know the community reasonably well, it's a lot of peer-to-peer acquired knowledge. 00:09:14.000 --> 00:09:26.000 Which is also great but the drawback might be that it's somewhat of a closed ecosystem i mean if i use a computing practice which is not great, and I'm sure I do. 00:09:26.000 --> 00:09:31.000 Then I pass it on to other people. They use the same thing. 00:09:31.000 --> 00:09:36.000 And we don't force credit. So that's not great. Which brings us to pray. 00:09:36.000 --> 00:09:39.000 And there are a lot to do? 00:09:39.000 --> 00:09:44.000 I'm sorry, answer the question. 00:09:44.000 --> 00:09:53.000 Okay, I don't know. Anyway, right. I mean, we can discuss all of this. 00:09:53.000 --> 00:10:12.000 Atlas, again, I think it's the same for all the big collaborations. We have what we call an induction day where the new students or collaborators joining are kind of introduced to the to the collaboration and there's a computing tutorial which goes on, I believe, for four days. So that's super helpful, very exhaustive. 00:10:12.000 --> 00:10:17.000 And we ask people, especially for people who are relative newcomers. 00:10:17.000 --> 00:10:22.000 Have you done those? And turns out only 60% have done that. 00:10:22.000 --> 00:10:26.000 So that's again i mean that's a good number, but it should ideally be close to 100%. 00:10:26.000 --> 00:10:34.000 If you are joining the collaboration, I think going through this is super useful So that's one thing which kind of bothered me. 00:10:34.000 --> 00:10:41.000 And then I asked them, are you part of an ongoing atlife analysis? Because maybe some of them are engineers. I mean, I don't know the identity of the people who replied. 00:10:41.000 --> 00:10:58.000 So maybe they don't really need to know all of these things almost 90%, I believe said They are involved in atlas analysis, so they must do the induction than the tutorial. So that's one thing which came out which is I think, is a good input for the community. 00:10:58.000 --> 00:11:11.000 And then I asked them, do you use grid? I mean, anyone doing analysis should and they should do so that's good And okay, this is more of me thinking. 00:11:11.000 --> 00:11:27.000 So when you are doing an analysis in Atlas. Or I guess anywhere else you request a signal sample, like a simulator signal sample now And then, of course, the way is you make some plots and validate and essentially what the single sample actually does 00:11:27.000 --> 00:11:31.000 What you should do, so can we get a million of those events? 00:11:31.000 --> 00:11:50.000 Now, there is a software called Rivet. Some of you might be aware of it. It's kind of Like, yeah, there's an MCAT collaboration with Eastern there it's them and cider and John Butterworth. So yeah, a bunch of people i work very closely with them and i think rivet is great. 00:11:50.000 --> 00:11:57.000 Slightly advertisement for analyzing particle level simulation. So if you haven't done it, look it up. 00:11:57.000 --> 00:12:12.000 And I find it great. But a large section in the community apparently uses like the whole atlas analysis yet, which is fine, which is not incorrect but As I said, I personally felt a bit let down. So I was trying to see what I can do 00:12:12.000 --> 00:12:21.000 And sometimes you end up with files in tech Which is like a more static storage. Do you know, I asked them, do you know what to do it? 00:12:21.000 --> 00:12:31.000 Most of them said no. We use something called R2D2, very creatively named to retrieve the files that run over it. So again, that's important. 00:12:31.000 --> 00:12:37.000 So then we asked at the end, what are the softwares used on your laptop? Of course, Root and Pilot. 00:12:37.000 --> 00:12:43.000 Are like the dominant ones, but also people use generators like Pythia and MedGraph. 00:12:43.000 --> 00:12:58.000 River is used by quite a few, not enough. Hopefully the one in the GN4. And someone wanted to use the Aclus, like the whole athena package, which is that a lot of software on their Mac, which I don't think is recommended, which is I don't think is optimal 00:12:58.000 --> 00:13:16.000 And then very quickly, am I running out of time? No, I have a couple of minutes. So if you're running machine learning, where do you run it? Most of them or one third of them run it on their laptop and one third is in like online platforms, which is okay. I think that's an important data point 00:13:16.000 --> 00:13:25.000 Then we asked them, do you want to get involved in Atlas computing development, which I think they would that's useful. Most of them said no. 00:13:25.000 --> 00:13:31.000 But then they use GPUs inexpress. Okay, that's my last slide. 00:13:31.000 --> 00:13:43.000 So before the pandemic, we actually used to have like a community workshop where all the Atlas people put together. No, not just Atlas. Atlas, at least we have Atlas and at least. 00:13:43.000 --> 00:13:48.000 All the athletes and artists, students would present. We'd have tutorials and stuff. 00:13:48.000 --> 00:13:54.000 It stopped during pandemic, unfortunately, due to many reasons. I'm not going to go into that. We never restarted it. 00:13:54.000 --> 00:14:16.000 Which is a pity. But I think looking at these things, we realized that there is a large scope of like training for younger students because that's our that's a huge part of our So we should organize local training events because many of these things can be optimized. And then I just want to end by saying that 00:14:16.000 --> 00:14:32.000 As I kind of said at the beginning, I mean, local realities play a big role. I mean, we are all sound poll about voters. We all have the same resources but whether people know that resources exist And this is not just distance. I think it's also like the local culture um 00:14:32.000 --> 00:14:49.000 Lack of training or lack of it All of this is important. So, I mean, I'm very happy I went through this this kind of hopefully we can find some ways in the community to improve the culture and maybe similar exercises in other countries, other smaller countries, I would say the developing countries would be useful 00:14:49.000 --> 00:14:55.000 But that's all I have. Thank you very much. 00:14:55.000 --> 00:15:05.000 Yeah, that was really clear and it was really nice to know the efforts specifically in Atlas Chandra Modeling, please. 00:15:05.000 --> 00:15:08.000 Am I audible? Thank you for the talk. 00:15:08.000 --> 00:15:10.000 Yes. 00:15:10.000 --> 00:15:27.000 So seeing the experiences that you've had Would you say that is it more or less easier to switch to more greener practices for a group that is just starting such as in South Africa. 00:15:27.000 --> 00:15:48.000 Because I think that well-established group might have be habitual to using more non-grainer methods And as you said that newer groups might have not access to these practices, these more eco-friendly practices So if you were like to say, or what do you think? 00:15:48.000 --> 00:15:51.000 Is it more or less easier? 00:15:51.000 --> 00:16:14.000 For that question. So I think that I've made clear two points, which one of them i should have mentioned that South Africa, unfortunately, our energy I mean, it's mostly Cool, which is not great but not great As you can imagine, we have zero control over it. In fact, people who know South Africa would know that we had like 00:16:14.000 --> 00:16:29.000 Lot of energy crises like huge power cuts in the last few years. It has become better in the last few years so And also the group The community is small and there is a lot of turnover in terms of students because we have a lot of math students who just do masters. 00:16:29.000 --> 00:16:37.000 A lot of students who just do PhDs and master's and PhDs and stuff But the group actually started in 2008, 9-ish. 00:16:37.000 --> 00:16:51.000 Even maybe earlier. So, I mean, we have a long culture it's just I think it has grown in the last few years. But yeah, coming back to your question i think yeah Because there is a large turnover and a large fraction of younger people 00:16:51.000 --> 00:17:00.000 We can actually If we do these trainings properly, I think that would be one of the focus of me and a few of our colleagues. 00:17:00.000 --> 00:17:18.000 Then we can we can actually do some of these things tell younger kids that do this, not that, at least in our small ways we can help. So yeah, thanks. 00:17:18.000 --> 00:17:42.000 Thank you very much. I see. No more questions. And you're also on time so thank you very much, Deepak, for sharing this um So now it's time for 15 minutes break. So we will try to reconvene in 15 minutes. So 10, 10 CST. 00:17:42.000 --> 00:18:12.000 Just bear in mind that we are potentially on time because there is uh one talk which will not happen because the person is not feeling well So yeah, please take a coffee or take a break and we reconvene in 15 minutes.