Daily News Report

1---
2name: daily-news-report
3description: Scrapes content based on a preset URL list, filters high-quality technical information, and generates daily Markdown reports.
4argument-hint: [optional: date]
5disable-model-invocation: false
6user-invocable: true
7allowed-tools: Task, WebFetch, Read, Write, Bash(mkdir*), Bash(date*), Bash(ls*), mcp__chrome-devtools__*
8---
9 
10# Daily News Report v3.0
11 
12> **Architecture Upgrade**: Main Agent Orchestration + SubAgent Execution + Browser Scraping + Smart Caching
13 
14## Core Architecture
15 
16```
17┌─────────────────────────────────────────────────────────────────────┐
18│                        Main Agent (Orchestrator)                    │
19│  Role: Scheduling, Monitoring, Evaluation, Decision, Aggregation    │
20├─────────────────────────────────────────────────────────────────────┤
21│                                                                      │
22│   ┌─────────────┐    ┌─────────────┐    ┌─────────────┐    ┌─────────────┐     │
23│   │ 1. Init     │ → │ 2. Dispatch │ → │ 3. Monitor  │ → │ 4. Evaluate │     │
24│   │ Read Config │    │ Assign Tasks│    │ Collect Res │    │ Filter/Sort │     │
25│   └─────────────┘    └─────────────┘    └─────────────┘    └─────────────┘     │
26│         │                  │                  │                  │           │
27│         ▼                  ▼                  ▼                  ▼           │
28│   ┌─────────────┐    ┌─────────────┐    ┌─────────────┐    ┌─────────────┐     │
29│   │ 5. Decision │ ← │ Enough 20?  │    │ 6. Generate │ → │ 7. Update   │     │
30│   │ Cont/Stop   │    │ Y/N         │    │ Report File │    │ Cache Stats │     │
31│   └─────────────┘    └─────────────┘    └─────────────┘    └─────────────┘     │
32│                                                                      │
33└──────────────────────────────────────────────────────────────────────┘
34         ↓ Dispatch                          ↑ Return Results
35┌─────────────────────────────────────────────────────────────────────┐
36│                        SubAgent Execution Layer                      │
37├─────────────────────────────────────────────────────────────────────┤
38│                                                                      │
39│   ┌─────────────┐   ┌─────────────┐   ┌─────────────┐              │
40│   │ Worker A    │   │ Worker B    │   │ Browser     │              │
41│   │ (WebFetch)  │   │ (WebFetch)  │   │ (Headless)  │              │
42│   │ Tier1 Batch │   │ Tier2 Batch │   │ JS Render   │              │
43│   └─────────────┘   └─────────────┘   └─────────────┘              │
44│         ↓                 ↓                 ↓                        │
45│   ┌─────────────────────────────────────────────────────────────┐   │
46│   │                    Structured Result Return                 │   │
47│   │  { status, data: [...], errors: [...], metadata: {...} }    │   │
48│   └─────────────────────────────────────────────────────────────┘   │
49│                                                                      │
50└─────────────────────────────────────────────────────────────────────┘
51```
52 
53## Configuration Files
54 
55This skill uses the following configuration files:
56 
57| File | Purpose |
58|------|---------|
59| `sources.json` | Source configuration, priorities, scrape methods |
60| `cache.json` | Cached data, historical stats, deduplication fingerprints |
61 
62## Execution Process Details
63 
64### Phase 1: Initialization
65 
66```yaml
67Steps:
68  1. Determine date (user argument or current date)
69  2. Read sources.json for source configurations
70  3. Read cache.json for historical data
71  4. Create output directory NewsReport/
72  5. Check if a partial report exists for today (append mode)
73```
74 
75### Phase 2: Dispatch SubAgents
76 
77**Strategy**: Parallel dispatch, batch execution, early stopping mechanism
78 
79```yaml
80Wave 1 (Parallel):
81  - Worker A: Tier1 Batch A (HN, HuggingFace Papers)
82  - Worker B: Tier1 Batch B (OneUsefulThing, Paul Graham)
83 
84Wait for results → Evaluate count
85 
86If < 15 high-quality items:
87  Wave 2 (Parallel):
88    - Worker C: Tier2 Batch A (James Clear, FS Blog)
89    - Worker D: Tier2 Batch B (HackerNoon, Scott Young)
90 
91If still < 20 items:
92  Wave 3 (Browser):
93    - Browser Worker: ProductHunt, Latent Space (Require JS rendering)
94```
95 
96### Phase 3: SubAgent Task Format
97 
98Task format received by each SubAgent:
99 
100```yaml
101task: fetch_and_extract
102sources:
103  - id: hn
104    url: https://news.ycombinator.com
105    extract: top_10
106  - id: hf_papers
107    url: https://huggingface.co/papers
108    extract: top_voted
109 
110output_schema:
111  items:
112    - source_id: string      # Source Identifier
113      title: string          # Title
114      summary: string        # 2-4 sentence summary
115      key_points: string[]   # Max 3 key points
116      url: string            # Original URL
117      keywords: string[]     # Keywords
118      quality_score: 1-5     # Quality Score
119 
120constraints:
121  filter: "Cutting-edge Tech/Deep Tech/Productivity/Practical Info"
122  exclude: "General Science/Marketing Puff/Overly Academic/Job Posts"
123  max_items_per_source: 10
124  skip_on_error: true
125 
126return_format: JSON
127```
128 
129### Phase 4: Main Agent Monitoring & Feedback
130 
131Main Agent Responsibilities:
132 
133```yaml
134Monitoring:
135  - Check SubAgent return status (success/partial/failed)
136  - Count collected items
137  - Record success rate per source
138 
139Feedback Loop:
140  - If a SubAgent fails, decide whether to retry or skip
141  - If a source fails persistently, mark as disabled
142  - Dynamically adjust source selection for subsequent batches
143 
144Decision:
145  - Items >= 25 AND HighQuality >= 20 → Stop scraping
146  - Items < 15 → Continue to next batch
147  - All batches done but < 20 → Generate with available content (Quality over Quantity)
148```
149 
150### Phase 5: Evaluation & Filtering
151 
152```yaml
153Deduplication:
154  - Exact URL match
155  - Title similarity (>80% considered duplicate)
156  - Check cache.json to avoid history duplicates
157 
158Score Calibration:
159  - Unify scoring standards across SubAgents
160  - Adjust weights based on source credibility
161  - Bonus points for manually curated high-quality sources
162 
163Sorting:
164  - Descending order by quality_score
165  - Sort by source priority if scores are equal
166  - Take Top 20
167```
168 
169### Phase 6: Browser Scraping (MCP Chrome DevTools)
170 
171For pages requiring JS rendering, use a headless browser:
172 
173```yaml
174Process:
175  1. Call mcp__chrome-devtools__new_page to open page
176  2. Call mcp__chrome-devtools__wait_for to wait for content load
177  3. Call mcp__chrome-devtools__take_snapshot to get page structure
178  4. Parse snapshot to extract required content
179  5. Call mcp__chrome-devtools__close_page to close page
180 
181Applicable Scenarios:
182  - ProductHunt (403 on WebFetch)
183  - Latent Space (Substack JS rendering)
184  - Other SPA applications
185```
186 
187### Phase 7: Generate Report
188 
189```yaml
190Output:
191  - Directory: NewsReport/
192  - Filename: YYYY-MM-DD-news-report.md
193  - Format: Standard Markdown
194 
195Content Structure:
196  - Title + Date
197  - Statistical Summary (Source count, items collected)
198  - 20 High-Quality Items (Template based)
199  - Generation Info (Version, Timestamps)
200```
201 
202### Phase 8: Update Cache
203 
204```yaml
205Update cache.json:
206  - last_run: Record this run info
207  - source_stats: Update stats per source
208  - url_cache: Add processed URLs
209  - content_hashes: Add content fingerprints
210  - article_history: Record included articles
211```
212 
213## SubAgent Call Examples
214 
215### Using general-purpose Agent
216 
217Since custom agents require session restart to be discovered, use general-purpose and inject worker prompts:
218 
219```
220Task Call:
221  subagent_type: general-purpose
222  model: haiku
223  prompt: |
224    You are a stateless execution unit. Only do the assigned task and return structured JSON.
225 
226    Task: Scrape the following URLs and extract content
227 
228    URLs:
229    - https://news.ycombinator.com (Extract Top 10)
230    - https://huggingface.co/papers (Extract top voted papers)
231 
232    Output Format:
233    {
234      "status": "success" | "partial" | "failed",
235      "data": [
236        {
237          "source_id": "hn",
238          "title": "...",
239          "summary": "...",
240          "key_points": ["...", "...", "..."],
241          "url": "...",
242          "keywords": ["...", "..."],
243          "quality_score": 4
244        }
245      ],
246      "errors": [],
247      "metadata": { "processed": 2, "failed": 0 }
248    }
249 
250    Filter Criteria:
251    - Keep: Cutting-edge Tech/Deep Tech/Productivity/Practical Info
252    - Exclude: General Science/Marketing Puff/Overly Academic/Job Posts
253 
254    Return JSON directly, no explanation.
255```
256 
257### Using worker Agent (Requires session restart)
258 
259```
260Task Call:
261  subagent_type: worker
262  prompt: |
263    task: fetch_and_extract
264    input:
265      urls:
266        - https://news.ycombinator.com
267        - https://huggingface.co/papers
268    output_schema:
269      - source_id: string
270      - title: string
271      - summary: string
272      - key_points: string[]
273      - url: string
274      - keywords: string[]
275      - quality_score: 1-5
276    constraints:
277      filter: Cutting-edge Tech/Deep Tech/Productivity/Practical Info
278      exclude: General Science/Marketing Puff/Overly Academic
279```
280 
281## Output Template
282 
283```markdown
284# Daily News Report (YYYY-MM-DD)
285 
286> Curated from N sources today, containing 20 high-quality items
287> Generation Time: X min | Version: v3.0
288>
289> **Warning**: Sub-agent 'worker' not detected. Running in generic mode (Serial Execution). Performance might be degraded.
290 
291---
292 
293## 1. Title
294 
295- **Summary**: 2-4 lines overview
296- **Key Points**:
297  1. Point one
298  2. Point two
299  3. Point three
300- **Source**: [Link](URL)
301- **Keywords**: `keyword1` `keyword2` `keyword3`
302- **Score**: ⭐⭐⭐⭐⭐ (5/5)
303 
304---
305 
306## 2. Title
307...
308 
309---
310 
311*Generated by Daily News Report v3.0*
312*Sources: HN, HuggingFace, OneUsefulThing, ...*
313```
314 
315## Constraints & Principles
316 
3171.  **Quality over Quantity**: Low-quality content does not enter the report.
3182.  **Early Stop**: Stop scraping once 20 high-quality items are reached.
3193.  **Parallel First**: SubAgents in the same batch execute in parallel.
3204.  **Fault Tolerance**: Failure of a single source does not affect the whole process.
3215.  **Cache Reuse**: Avoid re-scraping the same content.
3226.  **Main Agent Control**: All decisions are made by the Main Agent.
3237.  **Fallback Awareness**: Detect sub-agent availability, gracefully degrade if unavailable.
324 
325## Expected Performance
326 
327| Scenario | Expected Time | Note |
328|---|---|---|
329| Optimal | ~2 mins | Tier1 sufficient, no browser needed |
330| Normal | ~3-4 mins | Requires Tier2 supplement |
331| Browser Needed | ~5-6 mins | Includes JS rendered pages |
332 
333## Error Handling
334 
335| Error Type | Handling |
336|---|---|
337| SubAgent Timeout | Log error, continue to next |
338| Source 403/404 | Mark disabled, update sources.json |
339| Extraction Failed | Return raw content, Main Agent decides |
340| Browser Crash | Skip source, log entry |
341 
342## Compatibility & Fallback
343 
344To ensure usability across different Agent environments, the following checks must be performed:
345 
3461.  **Environment Check**:
347    -   In Phase 1 initialization, attempt to detect if `worker` sub-agent exists.
348    -   If not exists (or plugin not installed), automatically switch to **Serial Execution Mode**.
349 
3502.  **Serial Execution Mode**:
351    -   Do not use parallel block.
352    -   Main Agent executes scraping tasks for each source sequentially.
353    -   Slower, but guarantees basic functionality.
354 
3553.  **User Alert**:
356    -   MUST include a clear warning in the generated report header indicating the current degraded mode.
357