Merge pull request #66 from kaya70875/refactor/improve-return-types-for-core-methods

Refactor/improve return types for core methods

Author: kaya70875

CHANGELOG.md

@@ -5,8 +5,16 @@ All notable changes to this project will be documented here.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 
 ## [2.4.0]
+### Added
+- Added a `FetchResult` type alias for fetch results that may contain `ChannelData`, `VideoTranscript`, `VideoComments`, or `DLSnippet` objects.
+- Added `--transcripts-only` and `--snippets-only` CLI fetch modes.
+
 ### Changed
 - Improved developer experience with returning empty list objects on some methods instead of `None`.
+- Changed return types and values for `fetch_transcripts`, `fetch_snippets` and `fetch_comments` to improve type hints.
+- Changed CLI comment flags: `--comments` and `--comments-only` now select the fetch mode, while `--max-comments` controls the number of comments per video.
+- Exporters, `PreviewRenderer`, and `channel_data_to_rows()` now accept any supported fetch result shape and normalize it internally.
+
 
 ## [2.3.2]
 ### Fixed

README.md

@@ -19,9 +19,9 @@ A python tool for fetching thousands of videos fast from a Youtube channel along
 - [Features](#features)
 - [Fetching Specific Channel Tabs (Videos / Shorts / Streams)](#fetching-specific-channel-tabs-videos--shorts--streams)
 - [Using Different Fetchers](#using-different-fetchers)
-- [Retreive Different Languages](#retreive-different-languages)
+- [Retrieve Different Languages](#retrieve-different-languages)
 - [Filtering](#filtering)
-- [Converting ChannelData to Rows](#converting-channeldata-to-rows)
+- [Converting Fetch Results to Rows](#converting-fetch-results-to-rows)
 - [SQLite Cache](#sqlite-cache)
 - [Failed Transcripts & Retry Behavior](#failed-transcripts--retry-behavior)
 - [Fetching Only Manually Created Transcripts](#fetching-only-manually-created-transcripts)
@@ -232,7 +232,7 @@ These options can be passed to any of the fetcher methods (`from_channel`, `from
 
 See below for examples of usages.
 
-## Retreive Different Languages
+## Retrieve Different Languages
 
 You can use the `languages` param to retrieve your desired language. (Default en)
 
@@ -314,9 +314,9 @@ ytfetcher channel TheOffice -m 50 -f json --min-views 1000 --min-duration 300 --
 
 ---
 
-## Converting ChannelData to Rows
+## Converting Fetch Results to Rows
 
-If you want a flat, row-based structure for ML workflows (Pandas, HuggingFace datasets, JSON/Parquet), you can use the helper in `ytfetcher.utils` to join transcript segments. Comments are only included if you fetched them with `fetch_with_comments` or `fetch_comments`.
+If you want a flat, row-based structure for ML workflows (Pandas, HuggingFace datasets, JSON/Parquet), you can use the helper in `ytfetcher.utils` to join transcript segments. It accepts any fetch result returned by the public API, including `ChannelData`, `VideoTranscript`, `VideoComments`, and `DLSnippet` lists.
 
 ```python
 from ytfetcher import YTFetcher
@@ -328,6 +328,8 @@ channel_data = fetcher.fetch_with_comments(max_comments=5)
 rows = channel_data_to_rows(channel_data, include_comments=True)
 ```
 
+When comments are available, pass `include_comments=True` to include comment text in the output rows.
+
 ---
 
 ## SQLite Cache
@@ -451,7 +453,7 @@ ytfetcher channel TEDx -f csv --manually-created
 
 ## Exporting
 
-Use the `BaseExporter` class to export `ChannelData` in **csv**, **json**, or **txt**:
+Use the exporter classes to export `ChannelData` or any other supported fetch result in **csv**, **json**, or **txt**.
 
 ```python
 from ytfetcher.services import JSONExporter # OR you can import other exporters: TXTExporter, CSVExporter
@@ -527,35 +529,38 @@ fetcher = YTFetcher.from_channel("TheOffice", max_results=5)
 comments = fetcher.fetch_comments(max_comments=20)
 ```
 
-This will return list of `Comment` like this:
+This will return a list of `VideoComments` objects like this:
 
 ```python
 [
-    Comment(
-        text='Comment one.',
-        like_count=20,
-        author='@author',
-        time_text='8 days ago'
+    VideoComments(
+        video_id='id1',
+        comments=[
+            Comment(
+                text='Comment one.',
+                like_count=20,
+                author='@author',
+                time_text='8 days ago'
+            )
+        ]
     )
-
-    ## OTHER COMMENT OBJECTS...
 ]
 ```
 
 ### Fetching Comments With CLI
 
 Fetching comments in `ytfetcher` with CLI is very easy.
 
-To fetch comments with transcripts you can use `--comments` argument:
+To fetch comments with transcripts you can use the `--comments` mode. Use `--max-comments` to choose how many comments to fetch per video:
 
 ```bash
-ytfetcher channel TheOffice -m 20 --comments 10 -f json
+ytfetcher channel TheOffice -m 20 --comments --max-comments 10 -f json
 ```
 
-To fetch only comments with metadata you can use `--comments-only` argument:
+To fetch only comments you can use the `--comments-only` mode:
 
 ```bash
-ytfetcher channel TheOffice -m 20 --comments-only 10 -f json
+ytfetcher channel TheOffice -m 20 --comments-only --max-comments 10 -f json
 ```
 
 ## Other Methods
@@ -571,13 +576,17 @@ data = fetcher.fetch_transcripts()
 print(data)
 ```
 
+`fetch_transcripts()` returns `list[VideoTranscript]`. Each item contains the `video_id` and that video's transcript segments.
+
 ### Fetch Snippets
 
 ```python
 data = fetcher.fetch_snippets()
 print(data)
 ```
 
+`fetch_snippets()` returns `list[DLSnippet]` with video metadata only.
+
 ---
 
 ## Proxy Configuration
@@ -672,6 +681,7 @@ ytfetcher channel TheOffice -m 20 --tab shorts -f json
 
 # Fetch from the Live/Streams tab
 ytfetcher channel TheOffice -m 20 --tab streams -f json
+```
 
 ### Fetching by Video IDs
 
@@ -693,13 +703,13 @@ ytfetcher search "AI Getting Jobs" -f json -m 25
 ### Using Webshare Proxy
 
 ```bash
-ytfetcher <CHANNEL_HANDLE> -f json --webshare-proxy-username "<USERNAME>" --webshare-proxy-password "<PASSWORD>"
+ytfetcher channel <CHANNEL_HANDLE> -f json --webshare-proxy-username "<USERNAME>" --webshare-proxy-password "<PASSWORD>"
 ```
 
 ### Using Custom Proxy
 
 ```bash
-ytfetcher <CHANNEL_HANDLE> -f json --http-proxy "http://user:pass@host:port" --https-proxy "https://user:pass@host:port"
+ytfetcher channel <CHANNEL_HANDLE> -f json --http-proxy "http://user:pass@host:port" --https-proxy "https://user:pass@host:port"
 ```
 ---
 
@@ -714,7 +724,7 @@ docker-compose build
 Use `docker-compose run` to execute your desired command inside the container.
 
 ```bash
-docker-compose run ytfetcher poetry run ytfetcher channel -c TheOffice -m 20 -f json
+docker-compose run ytfetcher poetry run ytfetcher channel TheOffice -m 20 -f json
 ```
 ---
 

docs/cli.md

@@ -20,7 +20,7 @@ YTFetcher comes with a simple CLI so you can fetch data directly from your termi
 ytfetcher -h
 ```
 
-YTFetcher supports three main commands:
+YTFetcher supports four main commands:
 
 - `channel` - Fetch data from a YouTube channel handle
 - `video` - Fetch data from custom video IDs
@@ -159,21 +159,37 @@ All commands support the following common options:
 
 ### Comment Options
 
-**`--comments <NUMBER>`**
+**`--comments`**
 
-- Fetch top N comments alongside transcripts and metadata
-- Example: `ytfetcher channel TheOffice -m 20 --comments 10 -f json`
-- This fetches top 10 comments for each video along with transcripts
+- Fetch comments alongside transcripts and metadata
+- Example: `ytfetcher channel TheOffice -m 20 --comments --max-comments 10 -f json`
+- This fetches up to 10 comments for each video along with transcripts
 
-**`--comments-only <NUMBER>`**
+**`--comments-only`**
 
-- Fetch only comments with metadata (no transcripts)
-- Example: `ytfetcher channel TheOffice -m 20 --comments-only 10 -f json`
+- Fetch only comments (no transcripts or metadata)
+- Example: `ytfetcher channel TheOffice -m 20 --comments-only --max-comments 10 -f json`
+
+**`--max-comments <NUMBER>`**
+
+- Maximum comments to fetch per video
+- Default: `20`
+- Used with `--comments` or `--comments-only`
+
+**`--transcripts-only`**
+
+- Fetch only transcript data, without metadata
+- Example: `ytfetcher channel TheOffice -m 20 --transcripts-only -f json`
+
+**`--snippets-only`**
+
+- Fetch only video metadata, without transcripts
+- Example: `ytfetcher channel TheOffice -m 20 --snippets-only -f json`
 
 **`--sort` <`top`, `new`>**
 
 - Sort comments with top or newest ones (default to `top`).
-- Example: `ytfetcher channel TheOffice -m 10 -c --sort new`
+- Example: `ytfetcher channel TheOffice -m 10 --comments --max-comments 10 --sort new`
 
 !!! Warning
     Comment fetching is resource-intensive. Performance depends on your internet connection and the volume of comments being retrieved.
@@ -286,12 +302,6 @@ This command only processes videos that:
 - Get credentials from [Webshare Dashboard](https://dashboard.webshare.io/proxy/settings)
 - Example: `ytfetcher channel TheOffice -f json --webshare-proxy-username "your_username" --webshare-proxy-password "your_password"`
 
-**`--http-timeout`**
-
-- HTTP request timeout in seconds
-- Default: `4.0`
-- Example: `ytfetcher channel TheOffice --http-timeout 6.0`
-
 **`--http-headers`**
 
 - Custom HTTP headers (Python dictionary format)
@@ -317,7 +327,7 @@ This command:
 ### Fetch Comments with Transcripts
 
 ```bash
-ytfetcher channel TheOffice -m 10 --comments 5 -f csv -o ./data
+ytfetcher channel TheOffice -m 10 --comments --max-comments 5 -f csv -o ./data
 ```
 
 This command:
@@ -352,7 +362,7 @@ This command uses Webshare proxy to avoid rate limits when fetching large amount
 ### Export Only Comments
 
 ```bash
-ytfetcher channel TheOffice -m 20 --comments-only 15 -f json --filename comments_only
+ytfetcher channel TheOffice -m 20 --comments-only --max-comments 15 -f json --filename comments_only
 ```
 
 This command fetches only comments (no transcripts) and saves them to `comments_only.json`.

docs/exporting.md

@@ -1,8 +1,8 @@
 # Exporting
 
-The exporting feature allows you to save channel data in **multiple formats for analysis, reporting, or integration with other tools.** `ytfetcher` supports three widely-used export formats to suit different use cases and preferences.
+The exporting feature allows you to save fetched data in **multiple formats for analysis, reporting, or integration with other tools.** `ytfetcher` supports three widely-used export formats to suit different use cases and preferences.
 
-Use the `BaseExporter` class to export `ChannelData` in **csv, json, or txt**:
+Use the exporter classes to export `ChannelData` or any other supported fetch result in **csv, json, or txt**. Results from `fetch_youtube_data()`, `fetch_with_comments()`, `fetch_transcripts()`, `fetch_snippets()`, and `fetch_comments()` are normalized internally before writing.
 
 ```py
 from ytfetcher.services import JSONExporter # OR you can import other exporters: TXTExporter, CSVExporter
@@ -56,4 +56,4 @@ class XMLExporter(BaseExporter):
         output_path = self._initialize_output_path(export_type='xml')
         # Your custom logic to convert self.channel_data to XML
         print(f"Exporting data to {output_path}")
-```
+```

docs/index.md

@@ -101,7 +101,7 @@ You can also preview this data using `PreviewRenderer` class from `ytfetcher.ser
 ```py
 from ytfetcher.services import PreviewRenderer
 
-channel_data = fetcher.fetch_youtube_data(max_comments=10)
+channel_data = fetcher.fetch_with_comments(max_comments=10)
 #print(channel_data)
 preview = PreviewRenderer()
 preview.render(data=channel_data, limit=4)
@@ -257,9 +257,9 @@ fetcher = YTFetcher.from_channel(
 )
 ```
 
-## Converting ChannelData to Rows
+## Converting Fetch Results to Rows
 
-If you want a flat, row-based structure for ML workflows (Pandas, HuggingFace datasets, JSON/Parquet), use the helper in `ytfetcher.utils` to join transcript segments. Comments are only included if you fetched them with `fetch_with_comments` or `fetch_comments`.
+If you want a flat, row-based structure for ML workflows (Pandas, HuggingFace datasets, JSON/Parquet), use the helper in `ytfetcher.utils` to join transcript segments. It accepts any fetch result returned by the public API, including `ChannelData`, `VideoTranscript`, `VideoComments`, and `DLSnippet` lists.
 
 ```python
 from ytfetcher import YTFetcher
@@ -271,6 +271,8 @@ channel_data = fetcher.fetch_with_comments(max_comments=5)
 rows = channel_data_to_rows(channel_data, include_comments=True)
 ```
 
+When comments are available, pass `include_comments=True` to include comment text in the output rows.
+
 ## Fetching Comments
 `ytfetcher` allows you fetch comments in bulk **with additional metadata and transcripts** or **just comments alone.**
 
@@ -328,7 +330,7 @@ To fetch comments alongside with transcripts and metadata you can use `fetch_wit
 
 ```py
 fetcher = YTFetcher.from_channel("TheOffice", max_results=5)
-comments = fetcher.fetch_with_comments(max_comments=10, sort='top') # or new if you want latest comments
+channel_data = fetcher.fetch_with_comments(max_comments=10, sort='top') # or new if you want latest comments
 ```
 
 This will simply fetch **top 10 comments for every video** alongside with transcript data.
@@ -358,18 +360,21 @@ fetcher = YTFetcher.from_channel("TheOffice", max_results=5)
 comments = fetcher.fetch_comments(max_comments=20)
 ```
 
-This will return list of `Comment` like this:
+This will return a list of `VideoComments` objects like this:
 
 ```py
 [
-    Comment(
-        text='Comment one.',
-        like_count=20,
-        author='@author',
-        time_text='8 days ago'
+    VideoComments(
+        video_id='id1',
+        comments=[
+            Comment(
+                text='Comment one.',
+                like_count=20,
+                author='@author',
+                time_text='8 days ago'
+            )
+        ]
     )
-
-    ## OTHER COMMENT OBJECTS...
 ]
 ```
 
@@ -468,4 +473,4 @@ RuntimeConfig.enable_verbose()
 # Progress bars (tqdm) will now appear in your console
 fetcher = YTFetcher.from_channel(channel_handle="ChannelName")
 data = fetcher.fetch_youtube_data()
-```
+```

docs/release-notes.md

@@ -2,6 +2,8 @@
 
 ## Latest Changes
 ### Added
+- Added a `FetchResult` type alias for fetch results that may contain `ChannelData`, `VideoTranscript`, `VideoComments`, or `DLSnippet` objects.
+- Added `--transcripts-only` and `--snippets-only` CLI fetch modes.
 - Available results are no longer lost when `IpBlocked` is raised mid-fetch — collected transcripts are returned instead of raising an exception.
 - Introduced a new `FetchOptions` data class for defining fetcher options like `languages`, `filters` etc.
 - Added a `--sort` argument for choosing **top or new** comments with CLI.
@@ -12,6 +14,9 @@
 - Added transient failure categorization to improve retry/caching decisions.
 
 ### Changed
+- `fetch_transcripts()`, `fetch_snippets()`, and `fetch_comments()` now return more precise list types instead of mixed or optional shapes.
+- CLI comment flags now separate fetch mode from comment count: use `--comments` or `--comments-only` with `--max-comments`.
+- Exporters, `PreviewRenderer`, and `channel_data_to_rows()` now accept any supported fetch result shape and normalize it internally.
 - Removed deprecated `Exporter` class.
 - No more **network requests in __init__**.
 - `YTFetcher` now initializes correct `BaseYoutubeDLFetcher` inside classmethods.
@@ -157,4 +162,4 @@
 
 ### Changed
 - Update docs for `get_metadata` method.
-- Change default httpx.Timeout value to **4.0** to **2.0**.
+- Change default httpx.Timeout value to **4.0** to **2.0**.