Here I will list the earliest points where you can gather the material and my recommendation for easy harvesting.

Titanium

• Earliest: Safe Shallows (Metal Salvage & Limestone)
  
• Best: Safe Shallows & Kelp Forest (Metal Salvage & Limestone)
  
• Large Deposits: Lost River

Copper

• Earliest: Safe Shallows (Limestone)
  
• Best: Kelp Forest Caves & Grassy Plateus (Limestone)
  
• Large Deposits: Southern Blood Kelp Forest

Silver

• Earliest: Kelp Forest (Sandstone)
  
• Best: Underwater Islands (Sandstone)
  
• Large Deposits: Jellyshroom Caves & Lost River

Gold

• Earliest: Kelp Forest (Sandstone)
  
• Best: Underwater Islands, Jellyshroom Caves & Sea Treader’s Path (Sandstone & Shale Outcrop)
  
• Large Deposits: Jellyshroom Caves & Lost River

Nickel

• Found in Lost River, both in single chunks and Large Deposits.

Lead

• Earliest: Kelp Forest (Sandstone)
  
• Best: Kelp Forest Caves & Grassy Plateus (Sandstone)
  
• Large Deposits: Lost River

Lithium

• Earliest: Mountain Island (Shale Outcrop)
  
• Best: Mushroom Forest (seafloor) & Jellyshroom Caves (Shale Outcrop)
  
• Large Deposits: Lost River

Magnetite

• Earliest: Jellyshroom Caves (seafloor)
  
• Best: Jellyshroom Caves & Lost River (seafloor)
  
• Large Deposits: Jellyshroom Caves

Quartz

• Earliest: Safe Shallows (caves)
  
• Best: Grassy Plateus (in the grass) & Underwater Islands
  
• Large Deposits: Lost River & Grand Reef

Diamond

• Earliest: Jellyshroom Caves & Mountain Island (Shale Outcrop)
  
• Best: Underwater Islands & Sea Treader’s Path (Shale Outcrop)

Ruby

• Earliest: Sparse Reef (caves)
  
• Best: Grand Reef (seafloor and caves) & Bulb Zone (caves)

Kyanite

• Earliest: Inactive Lava Zone (Large Deposit)
  
• Best: Inactive Lava Zone (Large Deposit), Lava Castle (caves)

Crystalline Sulfur

• Earliest: Lost River (seafloor, in poison pools)
  
• Best: Lost River (seafloor, walls), Lava Castle (walls)

Uraninite

• Earliest: Mountain Island (underwater, on the mountain walls)
  
• Best: Lost River & Deep Grand Reef (seafloor, walls)
  
• Large Deposits: Underwater Islands, Lost River & Deep Grand Reef

Readers' findings:

• TheJAY_ZA recommends huge cave system in Mountains for Rubies and Uraninite. Watch out for Reapers nearby!
  
• Sandvich found Nickel in the Deep Blood Kelp Zone, about 800 meters down (caves).

• For easy resource gathering in middle parts of the game I recommend these leviathan-free zones: Underwater Islands, Mushroom Forest and Jellyshroom Caves. Later on Lost River provides you with almost every resource you need.
  
  
• Caves usually have the most rare resources of the biome, but be careful not to get lost.
  
  
• If you need gold, diamonds and lithium early, head to Mountain Island. Remember to bring water, cured food and a first aid kit.
  
  
• Build a Scanner Room to help you find useful materials.
  
  
• Add Storage Modules to your Seamoth and Prawn Suit for extra storing space on long mining trips.
  
  
• Prawn Suit Claw Arm can pick up resources and break outcrops. Items will be stored in Storage Module.

Limestone
Limestone is a great source of Titanium, important material for crafting early game gear. It is very common in biomes near the surface and is usually found on seabed, walls and caves.

Materials:

• Titanium
  
• Copper

Recommended harvesting locations:

• Safe Shallows (easiest)
  
• Kelp Forest (small & medium predators)
  
• Grassy Plateus (requires Oxygen Tank, small & medium predators)

Sandstone Outcrop
Materials from Sandstone are crucial for crafting important components, such as computer chips and wiring kits.

Materials: Lead, Silver, Gold

• Lead
  
• Silver
  
• Gold

Recommended harvesting locations:

• Kelp Forest (easiest, small & medium predators)
  
• Grassy Plateus (requires Oxygen Tank, small & medium predators)
  
• Underwater Islands (requires Seamoth, medium predators)

Shale Outcrop
Contains materials for advanced tools and gear.

Materials:

• Lithium
  
• Gold
  
• Diamond

Recommended harvesting locations:

• Mountain Island (easiest if you don’t have Seamoth, usually found underwater on the walls of the island base, avoid Warpers)
  
• Underwater Islands (easiest if you have seamoth, islands have plenty of Shale Outcrops on them, avoid Bonesharks)
  
• Grand Reef & Deep Grand Reef (requires Prawn Suit or upgraded Seamoth, large predators)

Barnacle
Found on Reefback leviathans (not aggressive). Reefbacks spawn near the surface of plain and forest type biomes. They might have aggressive spiked plants on their back, so watch out.

Materials:

• Copper
  
• Silver

Recommended harvesting locations:

• Grassy Plateus (easiest)
  
• Bulb Zone
  
• Sparse Reef

Large Resource Deposits give a great amount of one resource. They can be harvested with Prawn Suit Drill Arm.

Large Deposits are useful in mid to late game when returning to shallow biomes isn’t needed, reducing backtracking for gathering materials. The best places to find large deposits are in Lost River and Lava Zones. Tree Cove in Lost River is the safest option and is full of all kinds of Large Deposits, while Lava Zones have Large Deposits scattered all over the place with plenty of predators (and even Leviathans) lurking around.

Following materials have Large Resource Deposits: Titanium, Copper, Silver, Gold, Nickel, Lead, Lithium, Magnetite, Quartz, Kyanite and Uraninite.

Good locations for harvesting a specific Large Deposits are listed in Mining Resource Locations.