Optimizing French ChatGPT is a major challenge for Linagora. In the face of fierce competition, the company is considering innovative and sustainable solutions. Committed to a determined approach, Linagora is exploring a sovereign cloud infrastructure for its linguistic assistant, Lucie, with the ambition to reduce costs without sacrificing performance.
The technological choices are centered around optimal graphic cards ensuring a desirable performance-to-price ratio. The selection of GPU devices such as the Nvidia A4000 allows for the efficient deployment of language models while maintaining a controlled budget. In parallel, Linagora is considering a multicloud architecture to ensure data sovereignty while responding to increasingly high volume demands.
Linagora’s Strategies for the Economic Deployment of its French ChatGPT
Linagora is turning towards an open-source virtual assistant, named Lucie. This language model, which relies on a sovereign cloud infrastructure, has an ambition: to compete with major market solutions like ChatGPT. The architecture of Lucie is based on 7 billion parameters, thus allowing for effective processing of user requests.
Cloud Infrastructure and Technological Choices
As part of its implementation, Linagora chooses to rely on the cloud of EDF, Exaion. This strategic choice allows to take advantage of RTX A4000 graphic cards, offering 16 GB of NVRAM, sourced from a supercomputer. This partnership with Exaion facilitates the creation of a testing infrastructure, essential for the development phase of Lucie.
The acquisition cost of an RTX A4000 is $1,500, a price well below that of Nvidia H100 cards, which can reach $25,000. This choice is justified by a performance/cost ratio deemed favorable for the project.
Performance Optimization and Cost Management
Linagora’s teams are looking to reduce inference costs while maximizing performance. The RTX A4000 card, with a thermal envelope of 140 watts, consumes less energy than the H100, which has a TDP of 350 watts. This dynamic results in significant savings on the energy bill.
Tests conducted by Linagora reveal that the A4000 card can process about 10 prompts per second, a speed considered insufficient for the company’s scalability ambitions. Therefore, a multicloud architecture seems necessary to meet the demand. Linagora plans to use several cloud services, including OVHcloud and Scaleway, to achieve its goals.
Future Perspectives with the Mamba Model
Linagora plans to implement Mamba-type models in the future, which allow for more efficient data processing. Unlike models based on transformers, Mamba filters out less relevant data, thus optimizing processing. This choice could offer a significant reduction in memory and graphics performance requirements.
Mamba architectures display notable advantages for executing AI models, allowing for a more flexible use of available resources. This approach could revolutionize technological management while removing the debate over the rigid choice of graphic cards.
Sovereignty Challenges and Regulatory Compliance
The development of Lucie fits within a context of digital sovereignty. Linagora favors certified infrastructures, such as those complying with SecnumCloud standards. This guarantees the isolation of data from extraterritorial regulations, like the American Cloud Act.
In view of the launch of Lucie, Linagora positions itself as a key player in the ecosystem of open source AI, while ensuring optimal compliance with security and digital responsibility challenges.
To perfect the user experience and ensure a competitive offering, the company will implement performance monitoring and continuous adaptation of its technological choices. This strategy, based on reducing operating costs, reflects a desire to make AI accessible and functional for a wide audience.
Frequently Asked Questions about Linagora’s Strategies to Reduce Costs of its French ChatGPT
What are Linagora’s main strategies for optimizing costs of its French ChatGPT?
Linagora relies on a multicloud architecture, using infrastructures like those of Exaion from EDF and OVHcloud, to select graphic cards offering the best performance-cost ratio.
How does Linagora plan to leverage the cloud to reduce costs?
The company chooses sovereign cloud solutions that allow the model to run without overloading hardware capacities, while maintaining a reduced environmental footprint.
What types of graphic cards does Linagora use for its Lucie model?
Linagora primarily uses Nvidia A4000 and L4 cards, which provide good performance at a competitive cost, while exploring other options to improve processing volume.
How does Linagora evaluate the cost-effectiveness of its hardware resources?
Benchmarks and performance tests allow Linagora to compare graphic cards based on their acquisition cost and their effectiveness in the inference tasks required by AI.
What are the advantages of a small-scale language model (SLM) for Linagora?
An SLM like Lucie provides more efficient query execution with less resource needs, while being capable of competing with other similarly-sized models on the market.
Why is Linagora turning to older generation GPUs like the A4000?
Older generation GPUs, while less powerful, offer an excellent price-quality ratio in terms of acquisition costs and energy consumption, making their use particularly judicious for the Lucie project.
How does Linagora address the issue of data sovereignty in its strategy?
Linagora chooses SecnumCloud certified cloud solutions, ensuring that data remains under the French legislative framework and is isolated from extraterritorial regulations, which is crucial for its users.
What will be the implications of the multicloud architecture on operating costs?
This architecture reduces the risks of congestion and improves processing capacity while ensuring flexibility that helps better manage operational costs and optimize the resources used.
Does Linagora plan to use new technologies to improve its costs in the future?
Yes, Linagora is considering integrating Mamba-type models that would allow for better resource management, significantly reducing inference time while optimizing performance.
