当前位置: 首页 > news >正文

Elasticsearch 开放推理 API 增加了对 IBM watsonx.ai Slate 嵌入模型的支持

作者:来自 Elastic Saikat Sarkar

使用 Elasticsearch 向量数据库构建搜索 AI 体验时如何使用 IBM watsonx™ Slate 文本嵌入。

Elastic 很高兴地宣布,通过集成 IBM watsonx™ Slate 嵌入模型,我们的开放推理 API 功能得以扩展,这标志着我们与 IBM watsonx 团队持续合作的一个重要里程碑。通过此公告,Elasticsearch 用户可立即、轻松地访问 IBM 的 Slate 系列模型,而 IBM watsonx 社区则可利用 Elasticsearch 全面的 AI 搜索工具和经过验证的向量数据库功能。

Elastic 的开放推理 API(open inference API) 现正式发布,它使你能够创建端点并使用来自 IBM watsonx™ 等提供商的机器学习模型。IBM® watsonx™ AI 和数据平台包括核心组件和 AI 助手,旨在使用可信数据扩展和加速 AI 的影响。该平台具有开源 Slate 嵌入模型(slate-125m、slate-30m),用于检索增强生成、语义搜索和文档比较,以及在可信企业数据上训练的Granite 系列 LLMs。

在本博客中,我们将解释在使用 Elasticsearch 向量数据库构建 Search AI 体验时如何使用 IBM watsonx™ Slate 文本嵌入。Elastic 现在支持使用这些文本嵌入,新的 semantic_text 字段默认对传入文本进行分块,以适应平台模型的标记限制。

先决条件和推理端点的创建

你需要一个 IBM Cloud® Databases for Elasticsearch 部署。你可以通过目录、Cloud Databases CLI 插件、Cloud Databases API 或 Terraform 来配置一个。成功设置帐户后,你应该会进入 IBM Cloud 主页。

然后,你可以使用 IBM Cloud 的托管服务模型配置 Kibana 实例,并按照以下步骤连接到 Databases for Elasticsearch 实例 -

  • 为你的 Elasticsearch 部署设置管理员密码。
  • 安装 Docker 以提取 Kibana 容器映像并将其连接到 Databases for Elasticsearch。

或者,如果你不想在本地运行 Kibana 或安装 Docker,则可以使用 IBM Cloud® Code Engine 部署 Kibana。有关详细信息,请参阅有关使用 Code Engine 部署 Kibana 并将其连接到 Databases for Elasticsearch 实例的文档。

生成 API 密钥

  • 转到 IBM watsonx.ai 云并使用你的凭据登录。你将进入欢迎页面。

  • 转到 API 密钥页面。
  • 创建 API 密钥。

Elasticsearch 中的步骤

使用 Kibana 中的 DevTools,使用 watsonxai 服务为 text_embedding 创建推理端点:

PUT _inference/text_embedding/ibm_watsonx_embedding
{"service": "watsonxai","service_settings": {"api_key": "<api_key>","url": "xxx.ml.cloud.ibm.com","model_id": "ibm/slate-30m-english-rtrvr","project_id": "<project_id>","api_version": "2023-05-29"}
}

成功创建推理端点后,你将收到以下响应:

{"inference_id": "ibm_watsonx_embedding","task_type": "text_embedding","service": "watsonxai","service_settings": {"model_id": "ibm/slate-30m-english-rtrvr","project_id": "<project_id>","url": "xxx.ml.cloud.ibm.com","api_version": "2023-05-29","dimensions": 384,"similarity": "dot_product","rate_limit": {"requests_per_minute": 120}},"task_settings": {}
}

生成嵌入 - generate embeddings

下面是为单个字符串生成 text_embedding 的示例:

POST _inference/text_embedding/ibm_watsonx_embedding
{"input": "Embed this text"
}

你将收到以下嵌入响应:

{"text_embedding": [{"embedding": [0.009380317,0.05560313,...,        0.03804972]}]
}

此外,我们来看一个 semantic_text 映射示例

创建一个包含 semantic_text 字段的索引:

PUT books
{"mappings": {"properties": {"title": {"type": "text"},"semantic_text": {"type": "semantic_text","inference_id": "ibm_watsonx_embedding"},"description": {"type": "text","copy_to": ["semantic_text"]}}}
}

将一些文档插入到创建的索引中:

POST /books/_bulk
{ "index": {"_id": "1" }}
{ "title": "The Art of Coding", "description": "A comprehensive guide to coding best practices.", "price": 39.99 }
{ "index": { "_id": "2" } }
{ "title": "Mysteries of the Universe", "description": "Exploring the unknown aspects of our universe.", "price":
24.50 }
{ "index": {"_id": "3" } }
{ "title": "The Journey Within", "description": "A deep dive into personal development.", "price": 18.75 }
{ "index": {"_id": "4" } }
{ "title": "The richest man in babylon", "description": "The Richest Man in Babylon by George S. Clason is a classic book that offers timeless financial wisdom through a series of parables set in ancient Babylon, one of history’s wealthiest cities. The book conveys its lessons through stories of Babylonian citizens who learn to achieve wealth, prosperity, and happiness by following basic financial principles. The central character, Arkad, who is the richest man in Babylon, shares his wisdom on wealth-building with friends and fellow citizens. He explains how he started as a poor scribe but grew rich by adhering to a set of practical rules, which are presented as the Seven Cures for a Lean Purse and the Five Laws of Gold. The Seven Cures for a Lean Purse emphasize essential steps for accumulating wealth: saving at least ten percent of earnings, controlling expenses to live below one’s means, making wise investments, protecting investments from loss, owning a home, ensuring future income, and continually increasing the ability to earn. The Five Laws of Gold focus on key principles such as accumulating money through savings, investing wisely, seeking advice from experienced investors, avoiding investments in unfamiliar areas, and steering clear of get-rich-quick schemes. The book uses various characters like Bansir, a chariot maker, and Kobbi, a musician, to illustrate common financial challenges, and they seek Arkad’s guidance to achieve financial security, learning the importance of discipline, thrift, and prudent investment. Other stories, such as those of Dabasir, who faces challenges repaying debts, and Rodan, who learns about wise lending, further reinforce these principles. The book emphasizes that wealth is not a matter of luck or fate but a result of adhering to sound financial practices and cultivating a mindset geared towards saving, investing, and learning continuously. Through its simple yet profound lessons, the book provides readers with practical advice on personal finance, helping them understand the value of money management, disciplined saving, and wise investment strategies to build and maintain wealth over time.", "price": 18.75 }

接下来,使用 semantic_text 运行查询:

GET books/_search
{"query": {"semantic" : {"field": "semantic_text","query": "maintain wealth over time"}}
}

你将收到以下查询的响应:

{...,"hits": {"max_score": 0.86763954,"hits": [{"_index": "books","_id": "4","_score": 0.86763954,"_source": {"semantic_text": {"inference": {"inference_id": "ibm_watsonx_embedding","model_settings": {"task_type": "text_embedding","dimensions": 384,"similarity": "dot_product","element_type": "float"},"chunks": [{"text": "The Richest Man in Babylon by George S. Clason is a classic book that offers timeless financial wisdom through a series of parables set in ancient Babylon, one of history’s wealthiest cities. The book conveys its lessons through stories of Babylonian citizens who learn to achieve wealth, prosperity, and happiness by following basic financial principles. The central character, Arkad, who is the richest man in Babylon, shares his wisdom on wealth-building with friends and fellow citizens. He explains how he started as a poor scribe but grew rich by adhering to a set of practical rules, which are presented as the Seven Cures for a Lean Purse and the Five Laws of Gold. The Seven Cures for a Lean Purse emphasize essential steps for accumulating wealth: saving at least ten percent of earnings, controlling expenses to live below one’s means, making wise investments, protecting investments from loss, owning a home, ensuring future income, and continually increasing the ability to earn. The Five Laws of Gold focus on key principles such as accumulating money through savings, investing wisely, seeking advice from experienced investors, avoiding investments in unfamiliar areas, and steering clear of get-rich-quick schemes. The book uses various characters like Bansir, a chariot maker, and Kobbi, a musician, to illustrate common financial challenges, and they seek Arkad’s guidance to achieve financial security, learning the importance of discipline, thrift, and prudent investment. Other stories, such as those of Dabasir, who faces challenges repaying debts, and Rodan, who learns about","embeddings": [-0.0012952854,0.054616664,...]},{"text": " home, ensuring future income, and continually increasing the ability to earn. The Five Laws of Gold focus on key principles such as accumulating money through savings, investing wisely, seeking advice from experienced investors, avoiding investments in unfamiliar areas, and steering clear of get-rich-quick schemes. The book uses various characters like Bansir, a chariot maker, and Kobbi, a musician, to illustrate common financial challenges, and they seek Arkad’s guidance to achieve financial security, learning the importance of discipline, thrift, and prudent investment. Other stories, such as those of Dabasir, who faces challenges repaying debts, and Rodan, who learns about wise lending, further reinforce these principles. The book emphasizes that wealth is not a matter of luck or fate but a result of adhering to sound financial practices and cultivating a mindset geared towards saving, investing, and learning continuously. Through its simple yet profound lessons, the book provides readers with practical advice on personal finance, helping them understand the value of money management, disciplined saving, and wise investment strategies to build and maintain wealth over time.","embeddings": [-0.025139397,0.054932922,0.09041961,...]}]}},"price": 18.75,"description": "The Richest Man in Babylon by George S. Clason is a classic book that offers timeless financial wisdom through a series of parables set in ancient Babylon, one of history’s wealthiest cities. The book conveys its lessons through stories of Babylonian citizens who learn to achieve wealth, prosperity, and happiness by following basic financial principles. The central character, Arkad, who is the richest man in Babylon, shares his wisdom on wealth-building with friends and fellow citizens. He explains how he started as a poor scribe but grew rich by adhering to a set of practical rules, which are presented as the Seven Cures for a Lean Purse and the Five Laws of Gold. The Seven Cures for a Lean Purse emphasize essential steps for accumulating wealth: saving at least ten percent of earnings, controlling expenses to live below one’s means, making wise investments, protecting investments from loss, owning a home, ensuring future income, and continually increasing the ability to earn. The Five Laws of Gold focus on key principles such as accumulating money through savings, investing wisely, seeking advice from experienced investors, avoiding investments in unfamiliar areas, and steering clear of get-rich-quick schemes. The book uses various characters like Bansir, a chariot maker, and Kobbi, a musician, to illustrate common financial challenges, and they seek Arkad’s guidance to achieve financial security, learning the importance of discipline, thrift, and prudent investment. Other stories, such as those of Dabasir, who faces challenges repaying debts, and Rodan, who learns about wise lending, further reinforce these principles. The book emphasizes that wealth is not a matter of luck or fate but a result of adhering to sound financial practices and cultivating a mindset geared towards saving, investing, and learning continuously. Through its simple yet profound lessons, the book provides readers with practical advice on personal finance, helping them understand the value of money management, disciplined saving, and wise investment strategies to build and maintain wealth over time.","title": "The richest man in babylon"}},...]}
}

结论

通过集成 IBM watsonx™ 文本嵌入,Elasticsearch Open Inference API 继续为开发人员提供增强的功能,以构建强大而灵活的 AI 驱动搜索体验。探索 watsonx.ai 提供的更多受支持的编码器基础模型。

Elasticsearch 与行业领先的 Gen AI 工具和提供商进行了原生集成。查看我们的网络研讨会,了解如何超越 RAG 基础知识,或构建可用于生产环境的应用程序 Elastic Vector Database。

要为你的用例构建最佳搜索解决方案,请立即开始免费云试用或在你的本地机器上试用 Elastic。

原文:Elasticsearch open inference API adds support for IBM watsonx.ai Slate embedding models - Search Labs

相关文章:

Elasticsearch 开放推理 API 增加了对 IBM watsonx.ai Slate 嵌入模型的支持

作者&#xff1a;来自 Elastic Saikat Sarkar 使用 Elasticsearch 向量数据库构建搜索 AI 体验时如何使用 IBM watsonx™ Slate 文本嵌入。 Elastic 很高兴地宣布&#xff0c;通过集成 IBM watsonx™ Slate 嵌入模型&#xff0c;我们的开放推理 API 功能得以扩展&#xff0c;这…...

如何搭建一个小程序:从零开始的详细指南

在当今数字化时代&#xff0c;小程序以其轻便、无需下载安装即可使用的特点&#xff0c;成为了连接用户与服务的重要桥梁。无论是零售、餐饮、教育还是娱乐行业&#xff0c;小程序都展现了巨大的潜力。如果你正考虑搭建一个小程序&#xff0c;本文将为你提供一个从零开始的详细…...

NFS搭建

NFS搭建 单节点安装配置服务器安装配置启动并使NFS服务开机自启客户端挂载查看是否能发现服务器的共享文件夹创建挂载目录临时挂载自动挂载 双节点安装配置服务器安装配置服务端配置NFS服务端配置Keepalived编辑nfs_check.sh监控脚本安装部署RsyncInofity 客户端 单节点安装配置…...

RNN与LSTM,通过Tensorflow在手写体识别上实战

简介&#xff1a;本文从RNN与LSTM的原理讲起&#xff0c;在手写体识别上进行代码实战。同时列举了优化思路与优化结果&#xff0c;都是基于Tensorflow1.14.0的环境下&#xff0c;希望能给您的神经网络学习带来一定的帮助。如果您觉得我讲的还行&#xff0c;希望可以得到您的点赞…...

Docker部署FastAPI实战

在现代 Web 开发领域&#xff0c;FastAPI 作为一款高性能的 Python 框架&#xff0c;正逐渐崭露头角&#xff0c;它凭借简洁的语法、快速的执行速度以及出色的类型提示功能&#xff0c;深受开发者的喜爱。而 Docker 容器化技术则为 FastAPI 应用的部署提供了便捷、高效且可移植…...

【Python数据分析五十个小案例】电影评分分析:使用Pandas分析电影评分数据,探索评分的分布、热门电影、用户偏好

博客主页&#xff1a;小馒头学python 本文专栏: Python数据分析五十个小案例 专栏简介&#xff1a;分享五十个Python数据分析小案例 在现代电影行业中&#xff0c;数据分析已经成为提升用户体验和电影推荐的关键工具。通过分析电影评分数据&#xff0c;我们可以揭示出用户的…...

Vue2学习记录

前言 这篇笔记&#xff0c;是根据B站尚硅谷的Vue2网课学习整理的&#xff0c;用来学习的 如果有错误&#xff0c;还请大佬指正 Vue核心 Vue简介 Vue (发音为 /vjuː/&#xff0c;类似 view) 是一款用于构建用户界面的 JavaScript 框架。 它基于标准 HTML、CSS 和 JavaScr…...

TMS FNC UI Pack 5.4.0 for Delphi 12

TMS FNC UI Pack是适用于 Delphi 和 C Builder 的多功能 UI 控件的综合集合&#xff0c;提供跨 VCL、FMX、LCL 和 TMS WEB Core 等平台的强大功能。这个统一的组件集包括基本工具&#xff0c;如网格、规划器、树视图、功能区和丰富的编辑器&#xff0c;确保兼容性和简化的开发。…...

Redis主从架构

Redis&#xff08;Remote Dictionary Server&#xff09;是一个开源的、高性能的键值对存储系统&#xff0c;广泛应用于缓存、消息队列、实时分析等场景。为了提高系统的可用性、可靠性和读写性能&#xff0c;Redis提供了主从复制&#xff08;Master-Slave Replication&#xf…...

logback动态获取nacos配置

文章目录 前言一、整体思路二、使用bootstrap.yml三、增加环境变量四、pom文件五、logback-spring.xml更改总结 前言 主要是logback动态获取nacos的配置信息,结尾完整代码 项目springcloudnacosplumelog&#xff0c;使用的时候、特别是部署的时候&#xff0c;需要改环境&#…...

KETTLE安装部署V2.0

一、前置准备工作 JDK&#xff1a;下载JDK (1.8)&#xff0c;安装并配置 JAVA_HOME 环境变量&#xff0c;并将其下的 bin 目录追加到 PATH 环境变量中。如果你的环境中已存在&#xff0c;可以跳过这步。KETTLE&#xff08;8.2&#xff09;压缩包&#xff1a;LHR提供关闭防火墙…...

[RabbitMQ] 保证消息可靠性的三大机制------消息确认,持久化,发送方确认

&#x1f338;个人主页:https://blog.csdn.net/2301_80050796?spm1000.2115.3001.5343 &#x1f3f5;️热门专栏: &#x1f9ca; Java基本语法(97平均质量分)https://blog.csdn.net/2301_80050796/category_12615970.html?spm1001.2014.3001.5482 &#x1f355; Collection与…...

aws服务--机密数据存储AWS Secrets Manager(1)介绍和使用

一、介绍 1、简介 AWS Secrets Manager 是一个完全托管的服务,用于保护应用程序、服务和 IT 资源中的机密信息。它支持安全地存储、管理和访问应用程序所需的机密数据,比如数据库凭证、API 密钥、访问密钥等。通过 Secrets Manager,你可以轻松管理、轮换和访问这些机密信息…...

Java设计模式笔记(一)

Java设计模式笔记&#xff08;一&#xff09; &#xff08;23种设计模式由于篇幅较大分为两篇展示&#xff09; 一、设计模式介绍 1、设计模式的目的 让程序具有更好的&#xff1a; 代码重用性可读性可扩展性可靠性高内聚&#xff0c;低耦合 2、设计模式的七大原则 单一职…...

Unity3d C# 实现一个基于UGUI的自适应尺寸图片查看器(含源码)

前言 Unity3d实现的数字沙盘系统中&#xff0c;总有一些图片或者图片列表需要点击后弹窗显示大图&#xff0c;这个弹窗在不同尺寸分辨率的图片查看处理起来比较麻烦&#xff0c;所以&#xff0c;需要图片能够根据容器的大小自适应地进行缩放&#xff0c;兼容不太尺寸下的横竖图…...

【es6进阶】vue3中的数据劫持的最新实现方案的proxy的详解

vuejs中实现数据的劫持,v2中使用的是Object.defineProperty()来实现的&#xff0c;在大版本v3中彻底重写了这部分&#xff0c;使用了proxy这个数据代理的方式&#xff0c;来修复了v2中对数组和对象的劫持的遗留问题。 proxy是什么 Proxy 用于修改某些操作的默认行为&#xff0…...

w~视觉~3D~合集3

我自己的原文哦~ https://blog.51cto.com/whaosoft/12538137 #SIF3D 通过两种创新的注意力机制——三元意图感知注意力&#xff08;TIA&#xff09;和场景语义一致性感知注意力&#xff08;SCA&#xff09;——来识别场景中的显著点云&#xff0c;并辅助运动轨迹和姿态的预测…...

IT服务团队建设与管理

在 IT 服务团队中&#xff0c;需要明确各种角色。例如系统管理员负责服务器和网络设备的维护与管理&#xff1b;软件工程师专注于软件的开发、测试和维护&#xff1b;运维工程师则保障系统的稳定运行&#xff0c;包括监控、故障排除等。通过清晰地定义每个角色的职责&#xff0…...

一文学习开源框架OkHttp

OkHttp 是一个开源项目。它由 Square 开发并维护&#xff0c;是一个现代化、功能强大的网络请求库&#xff0c;主要用于与 RESTful API 交互或执行网络通信操作。它是 Android 和 Java 开发中非常流行的 HTTP 客户端&#xff0c;具有高效、可靠、可扩展的特点。 核心特点 高效…...

自研芯片逾十年,亚马逊云科技Graviton系列芯片全面成熟

在云厂商自研芯片的浪潮中&#xff0c;亚马逊云科技无疑是最早践行这一趋势的先驱。自其迈出自研芯片的第一步起&#xff0c;便如同一颗石子投入平静的湖面&#xff0c;激起了层层涟漪&#xff0c;引领着云服务和云上算力向着更高性能、更低成本的方向演进。 早在2012年&#x…...

安宝特方案丨XRSOP人员作业标准化管理平台:AR智慧点检验收套件

在选煤厂、化工厂、钢铁厂等过程生产型企业&#xff0c;其生产设备的运行效率和非计划停机对工业制造效益有较大影响。 随着企业自动化和智能化建设的推进&#xff0c;需提前预防假检、错检、漏检&#xff0c;推动智慧生产运维系统数据的流动和现场赋能应用。同时&#xff0c;…...

为什么需要建设工程项目管理?工程项目管理有哪些亮点功能?

在建筑行业&#xff0c;项目管理的重要性不言而喻。随着工程规模的扩大、技术复杂度的提升&#xff0c;传统的管理模式已经难以满足现代工程的需求。过去&#xff0c;许多企业依赖手工记录、口头沟通和分散的信息管理&#xff0c;导致效率低下、成本失控、风险频发。例如&#…...

【解密LSTM、GRU如何解决传统RNN梯度消失问题】

解密LSTM与GRU&#xff1a;如何让RNN变得更聪明&#xff1f; 在深度学习的世界里&#xff0c;循环神经网络&#xff08;RNN&#xff09;以其卓越的序列数据处理能力广泛应用于自然语言处理、时间序列预测等领域。然而&#xff0c;传统RNN存在的一个严重问题——梯度消失&#…...

el-switch文字内置

el-switch文字内置 效果 vue <div style"color:#ffffff;font-size:14px;float:left;margin-bottom:5px;margin-right:5px;">自动加载</div> <el-switch v-model"value" active-color"#3E99FB" inactive-color"#DCDFE6"…...

tree 树组件大数据卡顿问题优化

问题背景 项目中有用到树组件用来做文件目录&#xff0c;但是由于这个树组件的节点越来越多&#xff0c;导致页面在滚动这个树组件的时候浏览器就很容易卡死。这种问题基本上都是因为dom节点太多&#xff0c;导致的浏览器卡顿&#xff0c;这里很明显就需要用到虚拟列表的技术&…...

OPENCV形态学基础之二腐蚀

一.腐蚀的原理 (图1) 数学表达式&#xff1a;dst(x,y) erode(src(x,y)) min(x,y)src(xx,yy) 腐蚀也是图像形态学的基本功能之一&#xff0c;腐蚀跟膨胀属于反向操作&#xff0c;膨胀是把图像图像变大&#xff0c;而腐蚀就是把图像变小。腐蚀后的图像变小变暗淡。 腐蚀…...

华硕a豆14 Air香氛版,美学与科技的馨香融合

在快节奏的现代生活中&#xff0c;我们渴望一个能激发创想、愉悦感官的工作与生活伙伴&#xff0c;它不仅是冰冷的科技工具&#xff0c;更能触动我们内心深处的细腻情感。正是在这样的期许下&#xff0c;华硕a豆14 Air香氛版翩然而至&#xff0c;它以一种前所未有的方式&#x…...

PAN/FPN

import torch import torch.nn as nn import torch.nn.functional as F import mathclass LowResQueryHighResKVAttention(nn.Module):"""方案 1: 低分辨率特征 (Query) 查询高分辨率特征 (Key, Value).输出分辨率与低分辨率输入相同。"""def __…...

android13 app的触摸问题定位分析流程

一、知识点 一般来说,触摸问题都是app层面出问题,我们可以在ViewRootImpl.java添加log的方式定位;如果是touchableRegion的计算问题,就会相对比较麻烦了,需要通过adb shell dumpsys input > input.log指令,且通过打印堆栈的方式,逐步定位问题,并找到修改方案。 问题…...

十九、【用户管理与权限 - 篇一】后端基础:用户列表与角色模型的初步构建

【用户管理与权限 - 篇一】后端基础:用户列表与角色模型的初步构建 前言准备工作第一部分:回顾 Django 内置的 `User` 模型第二部分:设计并创建 `Role` 和 `UserProfile` 模型第三部分:创建 Serializers第四部分:创建 ViewSets第五部分:注册 API 路由第六部分:后端初步测…...