> ## Documentation Index
> Fetch the complete documentation index at: https://genai.codewithsiva.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# 8.Hugging Face Models

> Integrate Hugging Face models using hosted Serverless APIs or run tiny models locally on your system.

**Main Concepts Covered**

* [1. Using Hosted Hugging Face Models (init\_chat\_model)](#1-using-hosted-hugging-face-models-init_chat_model)
* [2. Direct Initialization](#2-direct-initialization)
  * [2.1 Hosted Option](#hosted-option)
  * [2.2 Local Option](#local-option)
* [3. Practice Exercises](#3-practice-exercises)

## 1. Using Hosted Hugging Face Models (init\_chat\_model)

The recommended, modern way to use hosted Hugging Face models in LangChain is by using the uniform `init_chat_model` function with `model_provider="huggingface"`. This handles model initialization, connection to the serverless Inference API, and chat message formatting automatically.

> \[!IMPORTANT]
> **Specify `backend="endpoint"`**: By default, `init_chat_model(model_provider="huggingface")` attempts to run models *locally* (using `HuggingFacePipeline` under the hood). This will trigger a gated model error and attempt to download the entire model to your system. To query the serverless hosted Inference API instead, always set `backend="endpoint"`.

**Question:**
Initialize a Hugging Face chat model directly using LangChain's uniform `init_chat_model` function to query the hosted serverless API and query it.

**Plan & Steps:**

1. Install `langchain-huggingface` package: `pip install langchain-huggingface`.
2. Ensure you have your `HUGGINGFACEHUB_API_TOKEN` set in your `.env` file.
3. Import `init_chat_model` from `langchain.chat_models`.
4. Initialize the chat model using `init_chat_model`, specifying an open-access model (like `Qwen/Qwen2.5-7B-Instruct`), setting `model_provider="huggingface"`, and setting `backend="endpoint"`.
5. Pass a list of structured messages to the model and invoke it.

```python theme={null}
from langchain.chat_models import init_chat_model
from langchain_core.messages import HumanMessage, SystemMessage
from dotenv import load_dotenv

load_dotenv()

# 1. Initialize the hosted model directly using core abstractions
chat_model = init_chat_model(
    model="Qwen/Qwen2.5-7B-Instruct",
    model_provider="huggingface",
    backend="endpoint",  # Forces serverless API usage instead of local model download
    temperature=0.7,
    max_new_tokens=256
)

# 2. Define structured messages
messages = [
    SystemMessage(content="You are a helpful science assistant."),
    HumanMessage(content="Explain what gravity is in 10 words.")
]

# 3. Invoke
response = chat_model.invoke(messages)
print(response.content)
```

### Code Explanation:

* **`init_chat_model(model_provider="huggingface")`**: Dynamically initializes and returns a `ChatHuggingFace` wrapper around an inferred serverless `HuggingFaceEndpoint` instance, offering a seamless provider-agnostic experience.
* **`backend="endpoint"`**: Tells LangChain to execute queries via Hugging Face's serverless hosted Inference API instead of installing and running the model locally.

#### Running Locally with `init_chat_model`:

If you want to run the model locally using `init_chat_model`, you can simply change `backend="endpoint"` to `backend="local"` (or omit it entirely, since local is the default backend). Just like the other local approaches, this will download the model to your machine and requires the local packages (`transformers`, `torch`, `accelerate`) installed:

```python theme={null}
# Downloads and runs locally on your machine
chat_model_local = init_chat_model(
    model="Qwen/Qwen2.5-0.5B-Instruct",
    model_provider="huggingface",
    backend="local",
    temperature=0.7,
    max_new_tokens=100
)
```

<div align="right">[Back to Top ⬆️](#)</div>

## 2. Direct Initialization

If you want a single-class interface resembling standard classes like `ChatOpenAI`, you can use the `from_model_id()` factory constructor. This allows you to easily run open-source models either hosted in the cloud (Serverless API) or completely locally on your hardware.

### Hosted Option

Query the hosted model on Hugging Face's serverless API using the direct factory method.

**Question:**
Initialize `ChatHuggingFace` directly without manually creating an endpoint object using the `from_model_id()` factory method to run hosted serverless queries.

**Plan & Steps:**

1. Import `ChatHuggingFace` from `langchain_huggingface` and `HumanMessage` from `langchain_core.messages`.
2. Initialize `ChatHuggingFace.from_model_id` passing the model ID (`Qwen/Qwen2.5-7B-Instruct`) and setting `backend="endpoint"`.
3. Pass a human message to the model and invoke it.

```python theme={null}
from langchain_huggingface import ChatHuggingFace
from langchain_core.messages import HumanMessage
from dotenv import load_dotenv

load_dotenv()

chat_model_hosted = ChatHuggingFace.from_model_id(
    model_id="Qwen/Qwen2.5-7B-Instruct",
    backend="endpoint",  # Forces hosted API execution
    model_kwargs={"temperature": 0.7, "max_new_tokens": 100}
)
response_hosted = chat_model_hosted.invoke([HumanMessage(content="Hello hosted model!")])
print("Hosted Response:", response_hosted.content)
```

<div align="right">[Back to Top ⬆️](#)</div>

### Local Option

To run the model completely locally on your own CPU or GPU (offline and private), you can set `backend="local"`.

#### Prerequisites:

Because this downloads the model and runs it on your machine, you must install the machine learning libraries first:

```bash theme={null}
uv add transformers torch accelerate
```

**Question:**
Initialize a chat model locally using `ChatHuggingFace.from_model_id()` and query it offline.

**Plan & Steps:**

1. Import `ChatHuggingFace` from `langchain_huggingface` and `HumanMessage` from `langchain_core.messages`.
2. Initialize the local chat model by setting `model_id="Qwen/Qwen2.5-0.5B-Instruct"` (a tiny \~1 GB model that runs easily on standard hardware) and setting `backend="local"`.
3. Pass a human message to the model and invoke it.

```python theme={null}
from langchain_huggingface import ChatHuggingFace
from langchain_core.messages import HumanMessage

# Initialize the local model
chat_model_local = ChatHuggingFace.from_model_id(
    model_id="Qwen/Qwen2.5-0.5B-Instruct",
    backend="local"  # Runs model locally using transformers pipeline
)
response_local = chat_model_local.invoke([HumanMessage(content="Hello local model!")])
print("Local Response:", response_local.content)
```

### Code Explanation:

* **`ChatHuggingFace.from_model_id`**: A factory constructor that automatically creates the underlying wrapper based on the model ID.
* **`backend="endpoint"` vs. `backend="local"`**: Toggles between serverless cloud execution (no heavy libraries needed) and local execution using local GPU/CPU.

<div align="right">[Back to Top ⬆️](#)</div>

## 3. Practice Exercises

### Practice 1: Swapping to GPU

How do you configure a local pipeline run to execute on a CUDA-enabled GPU (NVIDIA) if one is available on your machine?

<Accordion title="Solution">
  When using `ChatHuggingFace.from_model_id()` locally, you can pass pipeline parameters directly inside `pipeline_kwargs`. Setting `device_map="auto"` automatically handles GPU allocation:

  ```python theme={null}
  chat_model_local = ChatHuggingFace.from_model_id(
      model_id="Qwen/Qwen2.5-0.5B-Instruct",
      backend="local",
      pipeline_kwargs={
          "device_map": "auto",  # Automatically utilizes CUDA GPU if available
          "max_new_tokens": 100
      }
  )
  ```
</Accordion>

<div align="right">[Back to Top ⬆️](#)</div>