1,624 questions
0
votes
0
answers
40
views
I am stuck at this error - Error Loading the Langachain pipeline 'field set'
I am stuck at this error - Error Loading the Langachain pipeline 'field set'.
How can i resolve this error?
Below i have mention my code ,
import os
from langchain_community.vectorstores import FAISS
...
Advice
4
votes
3
replies
62
views
Improve the RAG chatbot result
I am building a local RAG chatbot using LangChain and ChromaDB (PersistentClient). I’m encountering 'hallucinations' when the similarity search returns documents with a low relevance score. How can I ...
- 1
Advice
0
votes
3
replies
119
views
Private AI Chatbot
How to build an AI-powered chatbot for a company that can answer users' questions based on company-specific data, interact with users through natural question-and-answer conversations similar to other ...
- 1
Advice
1
vote
1
replies
37
views
G6e.24xlarge vs G7e.12xlarge EC2 Instance Recomendation
I am planning to deploy llama 3.3 70b(FP8) Model in my EC2 instance, and I am wondering which would be good for better performance, GPU memory utilization, and operational complexity?
I will be just ...
- 1
Best practices
1
vote
1
replies
33
views
How to bridge existing W3C Web Services to a Model Context Protocol (MCP) Server for LLM consumption?
I have a set of established Web Services (W3C) on IIS servers and I want to expose their functionalities to LLMs using the Model Context Protocol (MCP). What is the recommended architectural pattern ...
- 1
3
votes
0
answers
52
views
Jupyter kernel crashes when loading local HuggingFace .safetensors model with AutoModelForCausalLM
I am attempting to load a large local Hugging Face model (
./ltx-2-19b-dev.safetensors", .safetensors format) inside JupyterLab using transformers, but the kernel crashes immediately during ...
- 51.3k
0
votes
2
answers
39
views
Promptfoo: RAG-Metrics - context-faithfulness assertion requires string output from the provider
I'm planning to evaluate a fine-tuned LLM in the same RAG system as the base model.
Therefore, I set up a PromptFoo evaluation.
In the process, I came across an error that I just can't wrap my head ...
- 31
Tooling
1
vote
0
replies
48
views
Agent which parses website through its file and folder
I want to build an npm package which any business can install into their project which then parses it and create client tools which will mimic a real user on the website. The main idea is to help ...
- 35
Best practices
5
votes
8
replies
267
views
AI use in coding
This has probably been talked about to death on this site but I still wanted to hear thoughts from other devs regarding best practices in using LLMs as guiding tools in coding
I specifically avoid ...
- 1
Best practices
2
votes
1
replies
58
views
Should you use a base model or instruction tuned model when LoRA fine-tuning an LLM?
I am trying to learn how to fine-tune models with the Huggingface suite of libraries (Transformers, PEFT, and TRL). On the Huggingface Hub there are many models that have base and instruction-tuned ...
- 4,269
Tooling
2
votes
1
replies
111
views
Looking for a local AI model and server that can do Copilot Agent mode in VSCode
VSCode lets you access models served from Ollama. But in the Manage Models selection screen, only the models hosted by services like OpenAI, Claude, etc. show "Capabilities" that are ...
- 579
0
votes
1
answer
74
views
How to implement loglikelihood() for an MLX-based lm-evaluation-harness using mlx_lm?
I'm writing a custom lm-evaluation-harness model backend that runs inference through Apple's MLX framework via mlx_lm, so I can benchmark quantized LLMs natively on Apple Silicon without routing ...
0
votes
0
answers
56
views
ETL Migration to Databricks via LLM Transpilation
I have several ETL jobs from DataStage in .dsx format. I use a PowerShell script to automatically run the migration for a larger number of files. And one job migrates successfully, while another ...
- 377
Best practices
4
votes
0
replies
92
views
Building a RAG chatbot where users can upload their own documents mid-conversation — how are you guys handling this?
I'm building a RAG-based chatbot (using OpenAI + FastAPI + Weaviate) and I've hit a point where I need to let users upload their own documents (PDFs, DOCX, images, TXT) directly inside a chat thread ...
- 670
-1
votes
0
answers
41
views
LLM Local Inference error with HuggingFace model
import pdfplumber
import json
from ctransformers import AutoModelForCausalLM
class Extractor:
def __init__(self, model_name="Mistral-7B-Instruct-v0.3-Q4_K_M.gguf"):
self.text = &...
- 54