[arrow-avro] Refactor `schema.rs` and add `ArrowToAvroSchemaBuilder` for explicit Arrow to Avro schema construction

**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**

`arrow-avro` currently contains Arrow -> Avro schema logic in `schema.rs` that was originally built as a *writer convenience* for when an Avro schema is not provided. As such, it was developed to make a best effort attempt at synthesizing an `AvroSchema` from an Arrow `Schema` (plus some Arrow schema metadata like `avro.name` / `avro.namespace`).

As `arrow-avro` adoption grows (OCF files, SOE frames, Confluent/Apicurio framing), we increasingly need schema behavior that is:
- **Explicit** about whether we are using a real Avro schema vs inferring one,
- **Modular** so it’s more maintainable (today `schema.rs` is a large multi-purpose module),
- **Correct-by-construction** so downstream consumers don’t need to patch up inferred schemas or reimplement Avro schema editing.

Real-world pain points motivating this include:
- Schema inference from Arrow metadata alone can produce incorrect Avro schemas for nested named types (e.g. #8928: confusion between nested record *type name* and *field name*).
- Downstream consumers (e.g. DataFusion) want to apply column projection at the Avro schema level without reimplementing Avro-aware projection and metadata handling (see #8923).
- Tests/integration code often treat `{ Arrow schema + avro.name/avro.namespace }` as sufficient, but this is not reliable for all schemas, and brittle inference can break reader schema workflows.
- The Arrow -> Avro path needs clearer configuration points (null-union ordering, naming strategy for generated nested types, metadata passthrough policy, etc.), but these knobs are currently either implicit, crate-private, or spread across helpers.

**Describe the solution you'd like**

Refactor / enhance `schema.rs` so Arrow -> Avro schema behavior is **explicit, modular, and correct-by-construction**, with APIs that clearly distinguish between these three fundamental schema conversion functions:
1. **"Using the real schema"** (preferred for readers): consume the Avro writer schema (OCF header / schema registry / user-provided) and optionally transform it into a reader schema as needed (projection, evolution).
2. **"Inferring a schema (defaults)"** (writer convenience): synthesize Avro JSON from an Arrow `Schema` when no Avro schema JSON is provided / embedded.
3. **"Building an explicitly correct `AvroSchema` from an Arrow `Schema` (configured builder for users)"**: add an `ArrowToAvroSchemaBuilder` for constructing an `AvroSchema` from an Arrow `Schema` with explicit configuration knobs.

Below are additional details for the proposed solution:

**A) Introduce `ArrowToAvroSchemaBuilder`**

Add a public builder style API along these lines:

```rust
use arrow_schema::Schema as ArrowSchema;
use arrow_avro::schema::AvroSchema;

// minimal defaults (equivalent to today's best effort inference)
let avro: AvroSchema = ArrowToAvroSchemaBuilder::new(&arrow_schema).build()?;

// use a writer schema to inform configuration
let avro: AvroSchema = ArrowToAvroSchemaBuilder::from_writer_schema(&arrow_schema, &avro_writer_schema).build()?;

// explicit configuration
let avro: AvroSchema = ArrowToAvroSchemaBuilder::new(&arrow_schema)
.with_root_name("User")
.with_namespace("com.example")
.with_doc("Schema inferred from Arrow")
.with_nullability_order(Nullability::NullFirst)
.with_strip_internal_arrow_metadata(true)
.with_type_naming_strategy(TypeNamingStrategy::PathBased)
.with_passthrough_metadata_policy(PassthroughMetadataPolicy::Default)
.build()?;
````

Initial builder "with_" knobs that would help correctness and downstream use-cases:

* Root record identity:

* `with_root_name(...)` (default: `AVRO_ROOT_RECORD_DEFAULT_NAME` or Arrow `avro.name`)
* `with_namespace(...)` (default: Arrow `avro.namespace` if present)
* `with_doc(...)` (default: Arrow `avro.doc` if present)
* Nullability + unions:

* `with_nullability_order(Nullability::NullFirst|NullSecond)` (default `NullFirst`, aligning with Avro union-default constraints)
* Metadata behavior:

* `with_strip_internal_arrow_metadata(bool)` (defaults to current behavior)
* `with_passthrough_metadata_policy(...)` controlling how non-reserved Arrow metadata becomes Avro attributes (today there is logic for "passthrough metadata" that could become configurable)
* Naming strategy for generated nested named types (records/enums/fixed):

* `with_type_naming_strategy(...)` to guarantee deterministic and collision-free nested type names
* (optional) `with_type_name_overrides(...)` for explicit mapping by Arrow field-path
* Logical/extension type policy:

* Define how Arrow logical/extension types map to Avro logical types, and what happens when unsupported (error vs fallback encoding)

This builder should be positioned as the explicit advanced inference entry point, while keeping a simpler defaults path for writer convenience.

**B) Make "use embedded schema" vs "infer schema" explicit**

Today, the Arrow schema metadata key `SCHEMA_METADATA_KEY = "avro.schema"` can contain the full Avro schema JSON. When present, it is often preferable to use it verbatim to preserve exact schema identity across OCF/SOE/registry contexts.

We should make this explicit and stable:

* A clear helper for "use embedded Avro schema if present, else error" (reader-like behavior)
* A clear helper for "use embedded schema if present, else infer" (writer convenience)

(Exact API design TBD, but could be builder flags or separate helpers.)

**C) Split `schema.rs` by responsibility (internal refactor)**

`schema.rs` currently mixes multiple concerns. Refactor into a module layout that preserves the public API but improves maintainability and testability, for instance:

* `schema::mod`: schema representation + serde + builder (Avro JSON)
* `schema::store`: schema store, canonical form + Rabin/MD5/SHA256 fingerprints
* `schema::metadata`: Arrow schema metadata keys + embed/extract helpers (`avro.schema`, `avro.name`, `avro.namespace`, `avro.doc`, defaults/enums)
* `schema::infer`: Arrow -> Avro inference logic (used by the builder)
* `schema::project`: Avro-aware projection/pruning utilities (ties into #8923)
* `schema::evolve`: Avro-aware evolution/extension utilities (also used by the builder)
* (optional) `schema::compat` / `schema::resolve`: compatibility checks + clearer error reporting (path + failure reason)

**D) Provide Avro-aware schema projection/evolution primitives**

Centralize Avro schema pruning/projection in `arrow-avro` (rather than downstream).
This is related to #8923 and would ideally live alongside the refactor so both "use real schema", "inference", and "builder" paths can share projection and evolution logic.

**E) Deprecate `AvroSchema::try_from()`**

Deprecate the existing `AvroSchema::try_from()` method and use `ArrowToAvroSchemaBuilder::new().build()?` in it's place. This shouldn't create any downstream behavior so long as `ArrowToAvroSchemaBuilder` matches `AvroSchema::try_from()` when no knobs are used.

**Describe alternatives you've considered**

1. **Continue fixing inference bugs incrementally without refactoring**

* Risks continued complexity growth in `schema.rs` and makes it harder to reason about correctness across reader/writer/projection paths.

2. **Require callers to always provide Avro schema JSON**

* This removes the writer convenience path and doesn’t address projection/evolution needs or tests where schemas are partially specified via Arrow metadata.

3. **Downstream projects implement Avro schema editing themselves**

* This duplicates Avro-specific logic and encourages subtle divergences from `arrow-avro` behavior, especially around naming, metadata, and resolution.

4. **Expose a single `InferOptions` struct instead of a builder**

* Works initially, but becomes less ergonomic as options grow, and makes it harder to evolve without breaking call-sites. A builder provides a more extensible surface.

**Additional context**

* Related issues:

* #8928 (nested named type: type name vs field name mismatch when generating schemas from Arrow-only metadata)
* #8923 (need Avro-aware projection API in `ReaderBuilder` / centralize schema editing)
* Relevant constants/metadata (current behavior to preserve where possible):

* `SCHEMA_METADATA_KEY = "avro.schema"`
* `AVRO_NAME_METADATA_KEY = "avro.name"`
* `AVRO_NAMESPACE_METADATA_KEY = "avro.namespace"`
* `AVRO_DOC_METADATA_KEY = "avro.doc"`
* `AVRO_FIELD_DEFAULT_METADATA_KEY = "avro.field.default"`
* `AVRO_ENUM_SYMBOLS_METADATA_KEY = "avro.enum.symbols"`
* Avro spec considerations that influence inference defaults:

* Union default values must match the first union branch, which is why `["null", T]` is typically preferred for optional fields:
[https://avro.apache.org/docs/1.11.1/specification/#unions](https://avro.apache.org/docs/1.11.1/specification/#unions)
* This issue is intentionally large: the goal is to land a design that solves the schema limitations in `arrow-avro` in the long-run. This will need to be implemented either via sub-issues or smaller partial PRs.

Start by reading arrow-avro's schema.rs and the related issues #8928 and #8923 to map the current inference, metadata, projection, and evolution responsibilities. The work is done when the design is split into implementable sub-issues with explicit APIs, preserved behavior where required, and focused validation for the builder and schema-handling paths.

Written by the indexing model from the issue text.