Deduplicate Instagram and TikTok exports by source identity, not by caption or engagement count. Keep the platform and stable post identifier or verified source URL together. Before removing a repeated row, decide whether it is an accidental duplicate or a useful observation of the same post at a different time.
Try the workflow in Chrome
Use FeedRama on a supported Instagram or TikTok page, and check the available range and actions in your account.
Add FeedRama to ChromeDefine what one row represents
If your sheet is a content inventory, one row may represent one post. If it tracks change, one row may represent one post observed on one date. Those are different data models. Removing repeats without deciding the model can erase the history you intended to analyze.
| Row meaning | Candidate key |
|---|---|
| One unique post | Platform plus verified post identity |
| One metric snapshot | Post identity plus collection date |
| One carousel slide | Post identity plus slide number |
| One research note | Post identity plus note context |
Do not match on captions
Creators can reuse captions, update wording, or publish similar content more than once. Two different posts with the same caption remain different posts. Two rows with different captions may still refer to one edited post.
Use the source URL as a starting point, then verify format differences before normalizing it. Do not remove meaningful path segments or turn photo-post URLs into video URLs merely to make them look consistent.
Reconcile duplicates in a working copy
- Preserve each raw export with its collection date.
- Add a platform column when combining Instagram and TikTok data.
- Group candidate duplicate identities and inspect them.
- Keep different dated observations when you are measuring change.
- For a unique inventory, choose and document which observation represents the post.
Use Instagram exports and TikTok exports as sources, not as automatic evidence that the merged sheet has unique posts.
Preserve the relationship to media
A carousel ZIP may contain many files but still correspond to one post-level metrics row. A second download of the same video does not create another post. Keep local filenames in a separate mapping if one source has multiple media files.
Use FeedRama and the download workflow for permitted media, then reconcile by source identity. Record how many rows you removed and why. A smaller sheet is useful only if it still represents the evidence you need.
Frequently asked questions
Can I remove duplicates by caption?
No. Captions can repeat or change. Match verified source identities and keep platform context.
Are two observations of one post duplicates?
They may be intentional snapshots from different dates. Keep them when studying changes over time.
Does each carousel slide count as a separate post?
Not for post-level engagement analysis. Track slides separately only when the analysis specifically needs slide-level records.
Should I normalize every URL automatically?
Verify the platform’s URL structure first. Removing meaningful segments can merge different resources or break source links.
What should I record after deduplication?
The identity rule, rows removed, observations retained, and the reason for choosing a representative row.