#AI #GenerativeAI #AIAgents #Science #AcademicPublishing #Reproducibility: "AI agents have the potential to aid users on a variety of consequential tasks, including conducting scientific research. To spur the development of useful agents, we need benchmarks that are challenging, but more crucially, directly correspond to real-world tasks of interest. This paper introduces such a benchmark, designed to measure the accuracy of AI agents in tackling a crucial yet surprisingly challenging aspect of scientific research: computational reproducibility. This task, fundamental to the scientific process, involves reproducing the results of a study using the provided code and data. We introduce CORE-Bench (Computational Reproducibility Agent Benchmark), a benchmark consisting of 270 tasks based on 90 scientific papers across three disciplines (computer science, social science, and medicine). Tasks in CORE-Bench consist of three difficulty levels and include both language-only and vision-language tasks. We provide an evaluation system to measure the accuracy of agents in a fast and parallelizable way, saving days of evaluation time for each run compared to a sequential implementation. We evaluated two baseline agents: the general-purpose AutoGPT and a task-specific agent called CORE-Agent. We tested both variants using two underlying language models: GPT-4o and GPT-4o-mini. The best agent achieved an accuracy of 21% on the hardest task, showing the vast scope for improvement in automating routine scientific tasks. Having agents that can reproduce existing work is a necessary step towards building agents that can conduct novel research and could verify and improve the performance of other research agents. We hope that CORE-Bench can improve the state of reproducibility and spur the development of future research agents."
Posts
-
#AI #GenerativeAI #AIAgents #Science #AcademicPublishing #Reproducibility: "AI agents have the potential to aid users on a variety of consequential tasks, including conducting scientific research. -
#AI #GenerativeAI #ClimateChange #CarbonEmissions #FossilFuels: "So the AI companies are exacerbating the climate crisis in at least three ways:#AI #GenerativeAI #ClimateChange #CarbonEmissions #FossilFuels: "So the AI companies are exacerbating the climate crisis in at least three ways:
- Selling tools to help oil companies AI tools to accelerate fossil fuel extraction.
- Running data centers that require ~10x the amount of electricity as Google
- Giving fossil fuel companies a reason—or an excuse—to build more power plants.
This makes it all the more pressing to deflate each of those bubbles. It’s a lot harder, after all, to decommission infrastructure than it is to not build it in the first place. And it should go without saying that, amid an accelerating climate crisis, we do not need to increase our energy use tenfold so that every tech giant can compete to shoddily automate call center and illustration jobs, inundate search results with tips for how to eat rocks safely, and load up our social feeds with dubious deepfakes.
The question is, how? The recent past offers one possible route. This isn’t the first time that the tech giants have been made to face scrutiny over their hypocritical climate policies—back in 2018, I wrote about the many ways that Google, Microsoft, and Amazon were selling AI and automation tools to fossil fuel companies. (The generative AI boom is like deja vu on steroids in this regard.) As the revelations of climate hypocrisy mounted, workers at those companies began public-facing pressure campaigns to get their employers to make good on their own climate promises. They made a climate-focused shareholder resolution, staged a public protest, and formed groups like the Amazon Employees for Climate Justice. The workers won concessions—Google said it would stop selling certain AI tools to oil companies, and Amazon made an elaborate if unenforceable Climate Pledge—even if they were far from what’s needed."
https://www.bloodinthemachine.com/p/ai-is-revitalizing-the-fossil-fuels
-
#LinkRot #DigitalPreservation #DigitalArchiving: "A quarter of all web pages that existed at some point between 2013 and 2023 now… don't.#LinkRot #DigitalPreservation #DigitalArchiving: "A quarter of all web pages that existed at some point between 2013 and 2023 now… don't. That's according to a recent study by Pew Research Center, a think tank based in Washington, DC, which raised the alarm of our disappearing digital history. Researchers found the problem is more acute the older a web page is: 38% of web pages that Pew tried to access that existed in 2013 no longer function. But it's also an issue for more recent publications. Some 8% of web pages published at some point 2023 were gone by October that same year.
This isn't just a concern for history buffs and internet obsessives. According to the study, one in five government websites contains at least one broken link. Pew found more than half of Wikipedia articles have a broken link in their references section, meaning the evidence backing up the online encyclopaedia's information is slowly disintegrating.
But thanks to the work of the Internet Archive, not all those dead links are totally inaccessible. For decades, the Archive's Wayback Machine project has sent armies of robots to crawl through the cascading labyrinths of the internet. These systems download functional copies of websites as they change over time – often capturing the same pages multiple times in a single day – and make them available to public free of charge.
"When we then went and looked at how many of those URLs were available in the Wayback Machine, we found that two-thirds of those were available in a way," he says. In that sense, the Internet Archive is doing what it set out to do – it's saving records of online society for posterity."
https://www.bbc.com/future/article/20240912-the-archivists-battling-to-save-the-internet
-
#EU #HumanRights #Greece #Surveillance #AsylumSeekers: "In May 2024, Access Now’s Caterina Rodelli travelled across Greece to meet with local civil society organisations supporting migrant people and monitoring human rights violations, and to see first...#EU #HumanRights #Greece #Surveillance #AsylumSeekers: "In May 2024, Access Now’s Caterina Rodelli travelled across Greece to meet with local civil society organisations supporting migrant people and monitoring human rights violations, and to see first-hand how and where surveillance technologies are deployed at Europe’s borders. In the first of a three-part blog series reflecting on what she saw, Caterina explains how, all too often, digitalising borders dehumanises the people trying to cross them."
https://www.accessnow.org/digital-surveillance-european-borders-part-1/
-
#AI #GenerativeAI #LinkedIn #AITraining #Privacy #DataProtection: "LinkedIn may have trained AI models on user data without updating its terms.@TimWardCam And I'm afraid they're right to think that...
-
#USA #China #Surveillance #Cisco #Privacy #FreedomOfSpeech #HumanRights: "EFF has long pushed companies that provide powerful surveillance tools to governments to take affirmative steps to avoid aiding and abetting human rights abuses. We have also wor...#USA #China #Surveillance #Cisco #Privacy #FreedomOfSpeech #HumanRights: "EFF has long pushed companies that provide powerful surveillance tools to governments to take affirmative steps to avoid aiding and abetting human rights abuses. We have also worked to ensure they face consequences when they do not.
Last week, the U.S. Court of Appeals for the Ninth Circuit helped this cause, by affirming its powerful 2023 decision that aiding and abetting liability in U.S. courts can apply to technology companies that provide sophisticated surveillance systems that are used to facilitate human rights abuses.
The specific case is against Cisco and arises out of allegations that Cisco custom-built tools as part of the Great Firewall of China to help the Chinese government target members of disfavored groups, including the Falun Gong religious minority. The case claims that those tools were used to help identify individuals who then faced horrific consequences, including wrongful arrest, detention, torture, and death."
https://www.eff.org/deeplinks/2024/09/human-rights-claims-against-cisco-can-move-forward-again
-
RT @RahllThose millions and millions in savings come from ripping off and absolutely screwing concept artists, storyboard artists, VFX artists, animators, matte painters, and so many others.RT @Rahll
Those millions and millions in savings come from ripping off and absolutely screwing concept artists, storyboard artists, VFX artists, animators, matte painters, and so many others.https://www.hollywoodreporter.com/business/business-news/lionsgate-deal-ai-firm-runway-1236005554/
-
#CyberSecurity #Encryption #Australia #Ghost: "Police in Australia have hacked into “Ghost,” an encrypted communications platform used by organized criminals, collected users’ messages, and arrested its alleged administrator, according to announcements...#CyberSecurity #Encryption #Australia #Ghost: "Police in Australia have hacked into “Ghost,” an encrypted communications platform used by organized criminals, collected users’ messages, and arrested its alleged administrator, according to announcements from various law enforcement agencies. Based on those messages, agencies across Europe, North America, and Australia have conducted raids in the last few days. Countries involved in the wider operation include Canada, France, Iceland, Ireland, Italy, the Netherlands, Sweden, and the United States.
The action is just the latest in law enforcement’s continued focus on the encrypted phone industry. The last large operation was Anom, in which the FBI secretly managed its own encrypted phone company to collect tens of millions of messages." https://www.404media.co/police-hack-into-ghost-an-encrypted-platform-for-criminals/
-
#CyberSecurity #FBI #Botnets #China #IoT #StateHacking: "Last week, the FBI took control of a botnet made up of hundreds of thousands of internet-connected devices, such as cameras, video recorders, storage devices, and routers, which was run by a Chin...#CyberSecurity #FBI #Botnets #China #IoT #StateHacking: "Last week, the FBI took control of a botnet made up of hundreds of thousands of internet-connected devices, such as cameras, video recorders, storage devices, and routers, which was run by a Chinese government hacking group, FBI director Christopher Wray and U.S. government agencies revealed Wednesday.
The hacking group, dubbed Flax Typhoon, was “targeting critical infrastructure across the U.S. and overseas, everyone from corporations and media organizations to universities and government agencies,” Wray said at the Aspen Cyber Summit cybersecurity conference on Wednesday.
“But working in collaboration with our partners, we executed court-authorized operations to take control of the botnet’s infrastructure,” Wray said, explaining that once the authorities did that, the FBI also removed the malware from the compromised devices. “Now, when the bad guys realized what was happening, they tried to migrate their bots to new servers and even conducted a [Distributed Denial of Service] attack against us.”"
-
#AI #GenerativeAI #LinkedIn #AITraining #Privacy #DataProtection: "LinkedIn may have trained AI models on user data without updating its terms."LinkedIn is using its users’ data for improving the social network’s generative AI products, but has not yet updated its terms of service to reflect this data processing, according to posts from various LinkedIn users and a statement from the company to 404 Media. Instead, the company says it will update its terms “shortly.”" https://www.404media.co/linkedin-is-training-ai-on-user-data-before-updating-its-terms-of-service/
-
#AI #GenerativeAI #LinkedIn #AITraining #Privacy #DataProtection: "LinkedIn may have trained AI models on user data without updating its terms.#AI #GenerativeAI #LinkedIn #AITraining #Privacy #DataProtection: "LinkedIn may have trained AI models on user data without updating its terms.
LinkedIn users in the U.S. — but not the EU, EEA, or Switzerland, likely due to those regions’ data privacy rules — have an opt-out toggle in their settings screen disclosing that LinkedIn scrapes personal data to train “content creation AI models.” The toggle isn’t new. But, as first reported by 404 Media, LinkedIn initially didn’t refresh its privacy policy to reflect the data use.
The terms of service have now been updated, but ordinarily that occurs well before a big change like using user data for a new purpose like this. The idea is it gives users an option to make account changes or leave the platform if they don’t like the changes. Not this time, it seems.
So what models is LinkedIn training? Its own, the company says in a Q&A, including models for writing suggestions and post recommendations. But LinkedIn also says that generative AI models on its platform may be trained by “another provider,” like its corporate parent Microsoft."
-
RT @AntonioCasilliGreat news!RT @AntonioCasilli
Great news! My book, "Waiting for Robots: The Hired Hands of Automation", is now available for pre-order. It's the culmination of years of research with my friends and colleagues, and despite the most recent data—plot twist—the robots are still a no-show.https://press.uchicago.edu/ucp/books/book/chicago/W/bo239039613.html#anchor-table-of-contents
-
RT @RonDeibertNEW: I'm *very excited* to announce my forthcoming book, "Chasing Shadows: Cyber Espionage, Subversion, and the Global Fight for Democracy," with @SimonSchusterCART @RonDeibert
NEW: I'm *very excited* to announce my forthcoming book, "Chasing Shadows: Cyber Espionage, Subversion, and the Global Fight for Democracy," with @SimonSchusterCAPublication date: Feb 4, 2025!
Details (including pre-order information) below:
https://simonandschuster.ca/books/Chasing-Shadows/Ronald-J-Deibert/9781668014042 -
#Crypto #Cryptocurrencies #PonziScheme: "An FBI report on internet crime in 2023 found there had been $4.57 billion in losses to investment frauds that year, with $3.96 billion of it (87%) being crypto-related."#Crypto #Cryptocurrencies #PonziScheme: "An FBI report on internet crime in 2023 found there had been $4.57 billion in losses to investment frauds that year, with $3.96 billion of it (87%) being crypto-related."
-
I'm sorry but this analysis is terribly simplistic."We’re entering a period where internet restrictions can’t just be easily dismissed as abusive actions taken by authoritarian governments, but one where they’re implemented by democratic states with the support of voting publics that are fed up with the reality of what the internet has become. They have no time for cyberlibertarian fantasies."
I almost forgot to mention this whole nonsense concept of "digital sovereignty". It's almost as if we've forgotten that sovereignty is always oppressive and antithetical to autonomy. I don't like sovereigns. What I'm fond of is of autonomous collectives and participatory rule-making
-
I'm sorry but this analysis is terribly simplistic.@dalonso Yes, I totally agree with that position.
-
I'm sorry but this analysis is terribly simplistic.I'm sorry but this analysis is terribly simplistic. Big Tech LOVES Intellectual Property, copyright, and specially patents. That's why these companies want to patent and extract licenses/subscriptions from everything that exists online. In the 90s, there were practically no large Internet companies (besides Yahoo!) and most technological infrastructure was decentralized - usenet newsgroups, IRC servers, BBSs, netizens movements, etc. Most technological innovations of that time were led by individual developers working for fun, as a hobby, or for mere survival.
While I'm sympathetic to leftist anti-big tech rhetoric, most of these discourses leave too many details out. You cannot create a false narrative based on misplaced assumptions, false narratives, and innuendos.
Digital rights were very important for the evolution of the Open Web and they should nowadays be considered as part of the human rights stack. Access to Knowledge (A2K) still continues to be critical for non-western countries, along with privacy and data protection rights.
Just because digital rights and the open-source movement were coopted by capitalism, that doesn't mean you should completely jettison them out. They are just a few pieces of a very large puzzle that must completed to surpass the conditions for the expansion of capitalism.
TL;DR: Leftists anti-big tech critics should read more marxist critical theory.
https://disconnect.blog/reclaiming-sovereignty-in-the-digital-age/
-
RT @WolfieChristlFollowing my 2021 investigation into digital profiling in the online gambling industry with @cleanupgambling and @A__W______O, the UK data protection authority ICO has confirmed that Sky Betting & Gaming's data processing was unlawful,...RT @WolfieChristl
Following my 2021 investigation into digital profiling in the online gambling industry with @cleanupgambling and @A__W______O
, the UK data protection authority ICO has confirmed that Sky Betting & Gaming's data processing was unlawful, breaching UK GDPR:
https://awo.agency/blog/ico-confirm-sky-bet-brands-acted-unlawfully/ -
#AI #GenerativeAI #Chatbots #ChatGPT #Medicine #Healthcare: "The survey, published in the journal BMJ Health and Care Informatics, spoke to 1,006 GPs. They were asked whether they had ever used any form of AI chatbot in their clinical practice, such as...#AI #GenerativeAI #Chatbots #ChatGPT #Medicine #Healthcare: "The survey, published in the journal BMJ Health and Care Informatics, spoke to 1,006 GPs. They were asked whether they had ever used any form of AI chatbot in their clinical practice, such as ChatGPT, Bing AI or Google’s Gemini, and were then asked what they used these tools for.
One in five of the respondents said that they had used generative AI tools in their clinical practice and, of these, almost a third (29%) said that they had used them to generate documentation after patient appointments, while 28% said that they had used the tools to suggest a different diagnosis.
A quarter of respondents said they had used the AI tools to suggest treatment options for their patients. These AI tools, such as ChatGPT, work by generating a written answer to a question posed to the software."
-
RT @1Br0wnExtraordinary: JD Vance says US could drop support for NATO if Europe tries to regulate M*sk’s platforms << is not going to succeed with its own exTwitter, but *could* greatly enable alternatives by requiring #interoperability! #DMA #Str...RT @1Br0wn
Extraordinary: JD Vance says US could drop support for NATO if Europe tries to regulate M*sk’s platforms << is not going to succeed with its own exTwitter, but *could* greatly enable alternatives by requiring #interoperability! #DMA #StrategicAutonomy