Compare commits

..

47 Commits

Author SHA1 Message Date
bryanthaboi fb97318e87 Merge pull request #1541 from bryanthaboi/dev
tuesday afternoon squashing
2026-08-18 17:08:46 -04:00
bryanthaboi 580449b8df Merge pull request #1538 from castdrian/ios-audio-stuff
fix(ios): recover audio after route changes
2026-08-18 17:02:10 -04:00
bryanthaboi fd73ab2a11 Merge branch 'dev' of https://github.com/bryanthaboi/gen1recomp into dev 2026-08-18 16:58:12 -04:00
bryanthaboi 48f710c3d6 Update bug_report.yml 2026-08-18 16:58:04 -04:00
bryanthaboi 794a5fc6cb Merge pull request #1540 from 1Jamie/fix/grass-standing-overdraw
fixes #1537 #1528  also fixes feet layering of gold with grass. Unifies gen1/gen2 grass layering into a single system
2026-08-18 16:55:39 -04:00
1jamie 286988a1e3 fixes #1537 #1528
also fixes feet layering of gold with grass. Unifies gen1/gen2 grass layering into a single system
2026-08-18 15:37:27 -05:00
Adrian Castro 48a3140a28 fix(ios): recover audio after route changes 2026-08-18 22:36:11 +02:00
bryanthaboi f7bdaa81f8 true color sprites shouldnt show in non color modes 2026-08-18 16:06:15 -04:00
bryanthaboi a1ab5e2cff title screen on gold going the wrong dir 2026-08-18 15:54:48 -04:00
bryanthaboi 99806ead25 Potentially CLOSES #1310, CLOSES #1005, CLOSES #1291 2026-08-18 15:49:47 -04:00
1jamie def967a8f8 fix(render): replace per-entity grass overdraw with full-screen cell pass
The previous drawCellBottom calls fired only for cells containing a
tracked entity.  While walking this worked acceptably because the
sprite's sub-pixel tween kept the visual overlap plausible, but while
standing still the sprite is pixel-aligned with the cell and the opaque
leaf-edge pixels in the grass bottom row paint over the player's feet.

This change removes the per-entity isGrassCell checks and replaces them
with a single post-sprite pass that overdraws every visible grass cell.

TileRenderer:
- ensureWindow now builds grassCells (all paths) and grassBatch (DMG/SGB
  shader path) alongside winBatch during the existing tile scan loop.
  A grassSeen table deduplicates cells so each cx/cy pair is only
  recorded once despite having two bottom-row tiles.
- drawGrassOverdraw: DMG/SGB draws the grassBatch under color0KeyShader
  in one call; GBC iterates grassCells and calls drawCellBottomRaw per
  cell (pre-keyed images can't share a SpriteBatch).
- markGrassOverdrawRedraw: iterates grassCells and calls
  markCellBottomRedraw for the post-zone OBP-replay pass (GBC only).
- releaseBatches cleans up grassBatch and grassCells.

OverworldController (flat path):
- Entity loop draws sprites only; grass overdraw fires once after the
  loop via drawGrassOverdraw + markGrassOverdrawRedraw.

OverworldController (tilt path):
- Grass cells are injected into the billboard sort queue keyed on the
  world-pixel foot of each cell's bottom tile row (cy*16+16), so they
  depth-sort correctly against entities at different y positions.  Each
  grass cell billboards via drawCellBottomRaw inside the upright pass.

Fixes: standing-in-tall-grass feet overdraw (Gen 2 confirmed, Gen 1
improved); NPCs and Pikachu follower in grass benefit automatically.
Parity test: tests/parity_grass_seam.lua 10/10, engine 228/228.
2026-08-18 13:23:10 -05:00
bryanthaboi 7583ba8729 CLOSES #1471 2026-08-18 14:19:42 -04:00
github-actions fc83ecd52f chore(ios): update app-repo.json [skip ci] 2026-08-18 12:05:11 -04:00
bryanthaboi 5ba49bdf3f Merge pull request #1524 from bryanthaboi/dev
squishing some fun ones
2026-08-18 11:55:28 -04:00
bryanthaboi abf95ce1c4 CLOSES #1519 2026-08-18 11:43:23 -04:00
bryanthaboi bd1046f398 CLOSES #1393 2026-08-18 11:29:50 -04:00
bryanthaboi d5ad830fb8 CLOSES #1418 2026-08-18 10:49:04 -04:00
bryanthaboi c2c7fdafcf CLOSES #1430 2026-08-18 10:36:44 -04:00
github-actions c01bda3570 chore(ios): update app-repo.json [skip ci] 2026-08-18 10:28:41 -04:00
bryanthaboi 74e04cb086 Merge pull request #1523 from bryanthaboi/dev
some bugs and some switch stuff
2026-08-18 10:18:58 -04:00
bryanthaboi 25ec896545 fix(audio): do not stop the move sfx source the replay just restarted
The takeover stop in Sound.playMove moved after the new source starts in
cfa84063, but an equal-id replay reuses the cached source, so the stop
killed the sound it had just restarted.
2026-08-18 10:14:44 -04:00
bryanthaboi 8240205254 ingested 2026-08-18 09:51:22 -04:00
bryanthaboi 55616fc03d CLOSES #1390 2026-08-18 09:20:07 -04:00
bryanthaboi 675971068e CLOSES #1503 2026-08-18 09:16:04 -04:00
bryanthaboi cfa8406306 CLOSES #1508 2026-08-18 04:52:00 -04:00
github-actions 0ea224d5db chore(ios): update app-repo.json [skip ci] 2026-08-17 23:13:43 -04:00
bryanthaboi c2e1db0f89 Merge pull request #1501 from bryanthaboi/dev 2026-08-17 23:03:49 -04:00
bryanthaboi 7b1e796c48 CLOSES #1496 2026-08-17 22:52:33 -04:00
github-actions 4c13770e70 chore(ios): update app-repo.json [skip ci] 2026-08-17 20:51:52 -04:00
bryanthaboi cf45cbbf92 Merge pull request #1498 from bryanthaboi/dev
more stuff
2026-08-17 20:41:58 -04:00
bryanthaboi 4e1ab1879b updated skin studio 2026-08-17 20:39:53 -04:00
bryanthaboi 917735a41c Merge branch 'dev' of https://github.com/bryanthaboi/gen1recomp into dev 2026-08-17 20:15:40 -04:00
bryanthaboi c877ea80a7 importer for skins now w file picker 2026-08-17 20:15:39 -04:00
github-actions 7d1ddf9b7c chore(ios): update app-repo.json [skip ci] 2026-08-17 20:09:51 -04:00
bryanthaboi faf82c2cec Merge pull request #1491 from AverageConsumer/codex/gen2-move-grid-hook
fix(gen2): honor mod move-grid navigation
2026-08-17 19:58:35 -04:00
bryanthaboi 5bc6036735 Merge pull request #1420 from anxiousintrovert/fix/required-import-platform-pickers
Fix required imports on Android legacy picker bridges
2026-08-17 19:58:13 -04:00
bryanthaboi 809628f0fa Merge pull request #1494 from AverageConsumer/codex/gen2-catch-previews
feat(gen2): expose stock ball catch previews
2026-08-17 19:57:37 -04:00
bryanthaboi 4817ff8bf9 Merge pull request #1492 from thibautbus/fix/museum-1f-ticket-clerk-strings
Route the museum 1F ticket clerk's remaining lines through Strings
2026-08-17 19:57:29 -04:00
anxiousintrovert 4f8739c029 Fix required imports on legacy Android picker bridges 2026-08-17 18:35:25 -05:00
AverageConsumer c655217120 feat(gen2): expose stock ball catch previews 2026-08-17 23:00:38 +02:00
thibautbus fbdfc1c053 Cover the museum 1F ticket clerk's translated lines with a targeted test 2026-08-17 21:44:43 +02:00
thibautbus 72c244b433 Route the museum 1F ticket clerk's remaining lines through Strings 2026-08-17 21:38:04 +02:00
AverageConsumer e0e030003b fix(gen2): honor mod move-grid navigation 2026-08-17 20:38:48 +02:00
github-actions ce2afb83f1 chore(ios): update app-repo.json [skip ci] 2026-08-17 14:13:48 -04:00
bryanthaboi 28f741f72f Merge pull request #1489 from bryanthaboi/dev
Update LauncherView.lua
2026-08-17 14:04:35 -04:00
bryanthaboi 22bcd95da1 Update LauncherView.lua 2026-08-17 14:03:07 -04:00
github-actions ea28f886f3 chore(ios): update app-repo.json [skip ci] 2026-08-17 13:20:07 -04:00
1327 changed files with 270595 additions and 148239 deletions
+16 -10
View File
@@ -5,6 +5,10 @@ body:
- type: markdown
attributes:
value: |
**Turn off all mods before filing.** Disable everything in the launcher's MODS
tab, confirm the bug still happens, then open this. Bugs that only show up with
mods on belong with the mod author, not here.
A screenshot is worth more than any description. If you can grab one, grab one.
If you genuinely can't, that's fine, but then the details below need to be thorough
enough that someone can find the bug without ever seeing your screen.
@@ -51,22 +55,24 @@ body:
validations:
required: true
- type: dropdown
id: mods_enabled
- type: checkboxes
id: mods_off
attributes:
label: Were any mods on
description: Check the MODS tab in the launcher if you're not sure.
label: Mods off
description: >
Turn off every mod in the launcher's MODS tab and reproduce the bug
before submitting. Do not file vanilla bugs with mods still enabled.
options:
- "No"
- "Yes"
validations:
required: true
- label: I turned off all mods and can still reproduce this
required: true
- type: input
id: mods_which
attributes:
label: Which mods (if any were on)
description: List the enabled mods. Leave blank if none were on.
label: Which mods (if you first noticed this with any on)
description: >
Optional. If you originally hit this with mods enabled, list them —
but only after you've confirmed it still happens with all of them off.
placeholder: nuzlocke 1.0.0, running-shoes 0.3
validations:
required: false
-118
View File
@@ -1,118 +0,0 @@
name: Feature request
description: Ask for something new in the engine, launcher, or platform — not a content/gameplay mod.
labels: ["enhancement"]
body:
- type: markdown
attributes:
value: |
Use this for **engine / launcher / platform** work (ports, video options, save
tooling, networking, mod API seams, docs).
If what you want is a gameplay, cosmetic, audio, or QoL change that a Lua mod
could ship — running shoes, alternate sprites, day/night, shiny indicators,
Gen 2-like battle toggles, soundtrack packs — open a
**[Mod request](https://github.com/bryanthaboi/gen1recomp/issues/new?template=mod_request.yml)**
instead.
"Can we add X" on its own is hard to act on. Say what you want, why you want it,
and how you picture it working.
- type: input
id: summary
attributes:
label: One line summary
description: What you want, in a sentence.
placeholder: Add Linux AppImage releases next to the macOS and Windows builds
validations:
required: true
- type: dropdown
id: game
attributes:
label: Which game is this about
description: Pick every version it applies to. Use N/A if it isn't game-specific.
multiple: true
options:
- Red
- Blue
- Yellow
- Gold
- N/A
validations:
required: true
- type: input
id: discord
attributes:
label: Discord username (optional)
description: >
So maintainers can ping you on Discord if they need a quick follow-up.
Leave blank if you'd rather keep everything on GitHub.
placeholder: yourname
validations:
required: false
- type: textarea
id: what
attributes:
label: What do you want
description: >
Describe it properly. What is it, where does it live (launcher, options,
engine), what does the player see or do. If it changes something that already
exists, say what it does today and what it should do instead.
placeholder: |
Ship a Linux AppImage on each release, same version as the macOS/Windows builds,
with the same save folder layout and mod discovery path.
validations:
required: true
- type: textarea
id: why
attributes:
label: Why is this worth doing
description: >
What's annoying or missing right now. What does this fix. If it's just because you
think it would be fun, say that, it's a real answer.
placeholder: |
LÖVE already runs on Linux; without a packaged build, players have to assemble
it themselves and miss release notes / update checks.
validations:
required: true
- type: textarea
id: how
attributes:
label: How should it work
description: >
The specifics. Which menu, what happens in the edge cases. If you don't
know, say what you'd expect as a player and leave the rest open.
placeholder: |
- GitHub Releases asset next to the .dmg / .exe
- Same options.lua / mods/ layout as desktop
- Documented in the README install section
validations:
required: true
- type: dropdown
id: scope
attributes:
label: Does this change how the original game plays
description: >
Some requests are quality of life, some change the actual game. Both are fine,
it just helps to know which one you're asking for.
options:
- Quality of life, original game is untouched
- Changes how the game plays
- Not sure
validations:
required: true
- type: textarea
id: extra
attributes:
label: Anything else
description: >
Reference screenshots, how another game does it, related issues. Leave blank
if nothing comes to mind.
validations:
required: false
-130
View File
@@ -1,130 +0,0 @@
name: Mod request
description: Ask for a gameplay, cosmetic, audio, or QoL change that belongs as a Lua mod.
labels: ["mod request"]
body:
- type: markdown
attributes:
value: |
This tracker is for ideas that should ship as **mods**, not as core engine
features — alternate sprites, running shoes, day/night, shiny indicators,
soundtrack packs, Gen 2-like battle toggles, map cosmetics, bag QoL, etc.
The engine already exposes a lot of this through registries and hooks
([modding wiki](https://github.com/bryanthaboi/gen1recomp/wiki)).
Use a **Feature request** instead for launcher / ports / video options /
networking / save tooling / new API seams.
- type: input
id: summary
attributes:
label: One line summary
description: What the mod should do, in a sentence.
placeholder: Hold B to run at bike speed on the overworld
validations:
required: true
- type: dropdown
id: game
attributes:
label: Which game is this for
description: Pick every version the mod should cover. Use N/A if it isn't game-specific.
multiple: true
options:
- Red
- Blue
- Yellow
- Gold
- N/A
validations:
required: true
- type: input
id: discord
attributes:
label: Discord username (optional)
description: >
So maintainers or mod authors can ping you on Discord if they pick this up.
Leave blank if you'd rather keep everything on GitHub.
placeholder: yourname
validations:
required: false
- type: textarea
id: what
attributes:
label: What should the mod do
description: >
Describe the player-facing behavior. What changes, where, what does the
player see or press. If it toggles from Options or a START-menu entry, say so.
placeholder: |
Hold B while walking outdoors to move at bike speed. Release to walk again.
Same places the bike is allowed; no effect in battles or menus.
validations:
required: true
- type: textarea
id: why
attributes:
label: Why is this worth doing as a mod
description: >
Why optional/modded rather than a core option. Who wants it on, who wants
vanilla left alone.
placeholder: |
Great for replaying and backtracking, but some people want a strict Gen 1
pace. A mod (or an opt-in mod option) keeps both camps happy.
validations:
required: true
- type: textarea
id: how
attributes:
label: How should it work
description: >
Buttons, menus, edge cases, whether it needs new art/audio. If you know a
hook or registry that fits (movement.speed, pokemon.sprite, rulesets, …),
mention it — otherwise leave it open.
placeholder: |
- Hold B on the overworld
- Same step timing as the bike
- Disabled where the bike is disabled
- Prefer hooks:wrap("movement.speed") if that still fits
validations:
required: true
- type: dropdown
id: vanilla
attributes:
label: With the mod off, is vanilla unchanged
options:
- Yes — parity when disabled
- No — it would replace something always-on
- Not sure
validations:
required: true
- type: dropdown
id: category
attributes:
label: Best-fit mod category
description: Same taxonomy as example mods (BALANCE, GRAPHICS, AUDIO, …).
options:
- GAMEPLAY / QoL
- GRAPHICS
- AUDIO
- BALANCE / ruleset
- CONTENT (maps, encounters, trainers)
- UI / TOOL
- TOTAL_CONVERSION-ish
- Not sure
validations:
required: true
- type: textarea
id: extra
attributes:
label: Anything else
description: >
Reference screenshots, other games/hacks that do it, related issues, or
"I'd like to try writing this myself." Leave blank if nothing comes to mind.
validations:
required: false
+4 -1
View File
@@ -120,7 +120,7 @@ jobs:
echo "changed=true" >> "$GITHUB_OUTPUT"
exit 0
fi
if git diff --name-only "$BASE_SHA" "$HEAD_SHA" | grep -Eq '^(scripts/build_switch\.sh$|scripts/switch/|docs/switch-.*\.md$|tests/switch_ci_workflows_test\.lua$|tests/switch_transfer_docs_test\.lua$|\.github/workflows/(ci|release|switch-artifact-comment)\.yml$|src/core/(NxAssetOverlay|Platform|GameVersion)\.lua$|src/import/CacheFs\.lua$|tests/engine/(assets_version_fallback|nx_generated_guard|nx_yellow_boot|switch_diagnostics)_test\.lua$|tests/engine/platform_nx)'; then
if git diff --name-only "$BASE_SHA" "$HEAD_SHA" | grep -Eq '^(scripts/build_switch\.sh$|scripts/switch/|docs/switch-.*\.md$|tests/switch_ci_workflows_test\.lua$|tests/switch_transfer_docs_test\.lua$|\.github/workflows/(ci|release|switch-artifact-comment)\.yml$|src/core/(NxAssetOverlay|Platform|GameVersion)\.lua$|src/import/CacheFs\.lua$|tests/engine/(assets_version_fallback|nx_generated_guard|nx_yellow_boot|switch_diagnostics|cache_fs_gold_nx_load)_test\.lua$|tests/engine/platform_nx)'; then
echo "changed=true" >> "$GITHUB_OUTPUT"
else
echo "changed=false" >> "$GITHUB_OUTPUT"
@@ -151,6 +151,9 @@ jobs:
luajit tests/engine/assets_version_fallback_test.lua
luajit tests/engine/nx_generated_guard_test.lua
luajit tests/engine/nx_yellow_boot_test.lua
luajit tests/engine/cache_fs_gold_nx_load_test.lua
luajit tests/engine/cache_fs_blue_mount_test.lua
luajit tests/engine/switch_diagnostics_test.lua
switch-build:
name: Switch fused build
+4 -9
View File
@@ -9,7 +9,7 @@ And before you say, "that's not a recomp", you're wrong. Recomp is an acronym. *
[Click Here for the AI Use Disclosure!](AIDisclosure.md)
> [!CAUTION]
> **We are NOT affiliated with the website `gen1recomp[.]com`** That website is not run by this project, was not authorized by us, and we have no idea who operates it. It is impersonating this project; do not download anything from it, and treat anything it hosts or claims as untrustworthy. Even if the site currently links back to this repository, the people behind it can change its content at any time, so nothing on it should ever be trusted. This GitHub repository and the Discord linked below are the only official sources for this project.
> **We are NOT affiliated with the website `gen1recomp[.]com`** That website is not run by this project, was not authorized by us, and we have no idea who operates it. It is impersonating this project; do not download anything from it, and treat anything it hosts or claims as untrustworthy. Even if the site currently links back to this repository, the people behind it can change its content at any time, so nothing on it should ever be trusted. This GitHub repository and the Discord linked below are the only official sources for this project. Also, as I assumed would eventually happen, the idiot that made that website now pumped it full of adware. Please stay away from that website.
<p align="center"><img src="https://raw.githubusercontent.com/bryanthaboi/gen1recomp/refs/heads/dev/assets/logo/logo.png"></p>
@@ -332,7 +332,7 @@ Maps can be edited in our own build of [Tiled](https://www.mapeditor.org),
and exported back out as a mod; see
[docs/tiled-map-editing.md](docs/tiled-map-editing.md).
## Bugs and Ideas
## Bugs
Found a bug? A warp dropping you somewhere it shouldn't, a battle doing math
that looks wrong, text in the wrong box, anything that does not match the
@@ -341,12 +341,6 @@ original game.
Attach a screenshot if you can. It saves a lot of back and forth, and if you
can't get one, the form asks you to describe what you saw instead.
Thought of a feature that could be good, or a way to improve one that already
exists?
[Open a feature request](https://github.com/bryanthaboi/gen1recomp/issues/new?template=feature_request.yml).
Say what you want, why it is worth doing, and how you picture it working. A
request with real detail is one that can actually get built.
## More
- [Link play](https://github.com/bryanthaboi/gen1recomp/wiki/Guide-Link-Play)
@@ -354,7 +348,8 @@ request with real detail is one that can actually get built.
- [Save editor](https://github.com/bryanthaboi/gen1recomp/wiki/Guide-Save-Editor)
— edit party, boxes, items, events, and Pokédex flags outside the game.
- `docs/architecture.md` — runtime details;
`docs/behavior-porting-notes.md` — formula provenance.
`docs/behavior-porting-notes.md` — formula provenance;
`docs/link-security.md` — what link play defends against, and what it doesn't.
+6 -3
View File
@@ -733,7 +733,8 @@ local function museumClerk(game, ow, done, onDecline)
local t = game.data.text or {}
if game.save.flags.EVENT_BOUGHT_MUSEUM_TICKET then
game.stack:push(TextBox.new(game,
"Take your time,\nand enjoy it all!", done))
t._Museum1FScientist1TakePlentyOfTimeText
or "Take your time,\nand enjoy it all!", done))
return
end
-- scripts/Museum1F.asm:72
@@ -751,10 +752,12 @@ local function museumClerk(game, ow, done, onDecline)
{ money = money }))
elseif yes then
game.stack:push(TextBox.new(game,
"You don't have\nenough money.", onDecline or done, { money = money }))
t._Museum1FScientist1DontHaveEnoughMoneyText
or "You don't have\nenough money.", onDecline or done, { money = money }))
else
game.stack:push(TextBox.new(game,
"Come again!", onDecline or done, { money = money }))
t._Museum1FScientist1ComeAgainText
or "Come again!", onDecline or done, { money = money }))
end
end }))
end
+120
View File
@@ -0,0 +1,120 @@
# Link play: threat model and what the code actually guarantees
Link play is the only part of this game that reads bytes written by
somebody else. This is what it defends against, what it does not, and
where each guarantee lives.
## The boundary
Everything a peer or the relay sends arrives as one JSON object per line.
There is exactly one place it becomes a message:
src/link/Net.lua reads bytes, frames lines, decodes JSON
src/link/Wire.lua rebuilds each line as a typed message
src/link/Session.lua the only path from a transport into a mode
`Session:update` runs `Wire.sanitize` on every message before anything
else sees it. A schema returns a **new** table holding only the fields it
names, at the Lua types it names, so the rest of `src/link/` can read
`msg.slot`, `msg.parts.actives` or `msg.mons[i].dvs.hp` directly and be
right by construction. A message with no schema (a mod's, or a future
build's) keeps a bounded, scalar-only copy of its payload instead of
being dropped.
A message that fails its schema is **dropped and logged**, never fatal.
Latching a terminal failure would hand a hostile peer a cheaper
disconnect than sending nothing at all.
### Why the bounds are loose
Wire's numeric bounds are deliberately wider than the game's own clamps in
`Protocol.unpackMon`. Both peers run identical clamps over identical
packets; a bound that bit an honest value would change one side's copy of
a mon and desync the lockstep. Wire's job is types and sizes. Rules are
`Protocol`'s job, and it keeps its own clamps for the callers that reach
it without a Session (the mod API, `tests/`).
### Containment behind it
Assume something still gets through:
- `Game:step` pcalls the link pump, and pcalls `stack:update` **only
while a link session is active**. On a throw, `Game:breakLink` closes
the connection, unwinds to the overworld and says "The link was
broken." Outside link play the stack is unguarded on purpose: a blanket
pcall would swallow real engine bugs and leave the game silently wrong
instead of loudly broken.
- `Net` caps `rxBuf` at 256KB and its per-frame read at 512KB, so a peer
that never sends a newline ends as a clean disconnect.
- `Json.decode` refuses documents nested past 64 levels, and takes an
optional length cap that the link path passes and the mod-manifest path
does not.
## The relay (`../pokeserver`)
- A line that is not a JSON **object** with a string `type` is dropped
before any handler runs, and `onLine` is wrapped in try/catch.
`server.js` installs `uncaughtException`/`unhandledRejection` handlers:
one bad packet must never take every live match down with the process.
- Line buffers are capped, lines per second are capped, connections per
IP and in total are capped, and an unbound connection that never hosts
or joins is swept after 30s.
- `SERVER_ONLY` is the set of message types the server is the only
legitimate author of (`peer_gone`, `bracket_update`, `match_start`,
`tournament_over`, `spectate`, ...). A peer that sends one has them
dropped rather than forwarded, so a bracket opponent cannot forge a
tournament result or fake "your opponent left".
- Trainer names are reduced to a printable subset and capped at the same
10 characters the game enforces, on the way in, because they are
rendered by the dashboard and broadcast to every participant.
`pokeserver/test/hostile.js` is the regression net for all of that.
## What is NOT defended
**Party legality is trust-the-client.** Online play meets strangers, and
`Handshake.onlineAllowed` is a Lua function in the same VM the mods load
into. It cannot be made tamper-proof in-process, and pretending otherwise
would only cost honest mod authors. What lockstep and
`Protocol.unpackMon`'s recompute-from-species-data *do* guarantee is that
a cheater cannot invent stats, moves, or a shiny: every derived value is
rebuilt locally from real species data. They can send a legal party they
farmed or edited. That is the honest boundary.
What the relay does instead is **observe and record**. It already sees
every `hello`, so it keeps each connection's self-reported
`engineVersion`, `fingerprint` and `linkModified`, compares the two sides
of a room or a live tournament match, and logs and surfaces a
`modded` / `fingerprint_mismatch` / `version_skew` flag on the dashboard.
A patched client can still lie; what it cannot do is lie without the
tournament organizer having a record of it.
Client-side attestation is deliberately not built. This is an
open-source Lua game: it would be theater, and it would break honest
mods.
**The relay has no TLS.** Port 7778 is plaintext, so party contents,
trades and trainer names are visible to anyone on the network path. There
is nothing secret in a Pokemon party, but it is a real property of the
system and not an oversight. Fixing it means a TLS terminator in front of
the relay and a client that speaks it, which is a version break for every
shipped build.
**The dashboard has no default password.** `DASHBOARD_PASSWORD` is
required; with it unset the relay runs and the dashboard simply does not
start. It is still Basic Auth over plain HTTP, so it belongs behind an
IP restriction or an SSH tunnel (`pokeserver/DEPLOY.md`).
## Tests
luajit tests/link_hostile.lua every message type x every wrong type
luajit tests/link_desync_fuzz.lua lockstep fuzz, plus a mutation mode
luajit tests/run_link_tests.lua both of the above, plus the rest
cd ../pokeserver && npm test relay smoke, 16-player bracket, hostile
`tests/link_hostile.lua` builds its corpus from a template per message
type, replaces each field (and several nested ones) with every wrong Lua
type, and drives the survivors through the real trade session, a real
lockstep battle, a real spectator battle, and the tournament screen
**including its draw** -- because the two nastiest payloads are
delayed-fuse ones that crash on render rather than on receipt.
+3 -2
View File
@@ -540,8 +540,9 @@ gains a field instead of the name gaining a prefix.
`battle.crit`, `battle.accuracy`, `battle.turn_order`,
`battle.enemy_action`, `battle.run`, `battle.exp_award`, `exp.gain`,
`catch.rate`, `trainer.party`, `battle.overlay`, `battle.low_health_alarm`,
`battle.catch_exp`, `battle.bottom_ui_visible` and
`battle.status_hud_visible`. One payload difference: Gen 1's vanilla
`battle.catch_exp`, `battle.bottom_ui_visible`,
`battle.status_hud_visible` and `battle.move_grid_navigation`. One payload
difference: Gen 1's vanilla
`battle.low_health_alarm` link reads `ctx.battle.data`, and Gold's battle
screen has no `.data` field, so the Gen 2 site **adds** `ctx.data` beside the
Gen 1 keys. A mod that calls `nextFn` is unaffected; one that reaches through
+4 -4
View File
@@ -268,10 +268,10 @@ an optional stock `catchChance` percentage.
`prompt` describes the currently visible choice (`menu`, `moves`, `party`,
`advance`, `safari`, or `mimic`) and is `locked` when another screen or battle
phase owns input. Generation-specific features remain optional: Gen 1 includes
battle medicine, balls, catch previews, Safari balls, and Mimic choices;
Gold currently returns an empty `items` list rather than guessing at its
pocketed PACK flow. Callers should ignore unknown fields and tolerate absent
optional ones.
battle medicine, balls, catch previews, Safari balls, and Mimic choices. Gold
exposes balls and their exact stock catch previews; targeted medicine remains
screen-owned and is omitted rather than guessing at its pocketed PACK flow.
Callers should ignore unknown fields and tolerate absent optional ones.
## Battle menu intents
+1
View File
@@ -13,6 +13,7 @@ Features intentionally added beyond the original Pokémon Red, Blue, and Yellow
* **Mobile touch controls** with editable layouts, vibration, and orientation settings
* **Touch skins** in RetroArch overlay format, with bezel art, per-button press states, and Super Game Boy borders
* **Pokédex diploma and printer image exports**
* **Mod download counts** from the index feed, with Most-downloaded and Trending sorts
## Gen 2 Specifics
+21 -5
View File
@@ -6,6 +6,7 @@ control layout, and the rectangle the Game Boy screen is drawn into. Engine:
(draw and input), `src/render/Renderer.lua` (the screen viewport),
`src/ui/SkinStudio.lua` (the desktop editor). Tests:
`tests/engine/touch_skin_test.lua`, `tests/engine/skin_studio_test.lua`,
`tests/engine/skin_studio_image_import.lua`,
`tests/engine/launcher_skins_tab.lua`.
Skins are picked in the launcher's **Skins** tab, which also imports them and
@@ -25,7 +26,7 @@ as-is. Supported keys:
| `overlayN_overlay` | bezel image |
| `overlayN_full_screen` | stretch the page to the window |
| `overlayN_rect` | page placement, default `0,0,1,1` |
| `overlayN_aspect_ratio` | fallback aspect when not full screen |
| `overlayN_aspect_ratio` | design aspect; the overlay letterboxes to it even when full screen |
| `overlayN_range_mod`, `overlayN_alpha_mod` | desc defaults |
| `overlayN_viewport` | `x,y,w,h`, the screen cutout |
| `overlayN_viewport_fill` | parsed; the engine always fits, see below |
@@ -159,8 +160,25 @@ The Super Game Boy preset locks the viewport to the real screen window,
resize. X / Y / W / H are in canvas pixels, so a control can be typed to the
coordinate its art was drawn at. Bind, hitbox shape, hit reach and idle and
pressed images are per control; the bezel, the pages and the screen cutout are
per page. The cutout is itself a draggable element with a 10:9 lock. Drop a PNG
or JPG on the window to import art into the skin.
per page. The cutout is itself a draggable element with a 10:9 lock.
Each page can **Lock** to portrait or landscape. With **Match canvas** on
(the default), Next page picks a matching mock device and the canvas preset
picks a matching page. Turn Match canvas off to look at a portrait page on a
landscape device.
A RetroArch overlay whose pages are already named portrait / landscape
(the auto-rotate convention) locks those pages and turns Match canvas on
when you open it. You do not have to click Lock first.
**Art.** The **Bezel**, **Idle art** and **Pressed art** rows cycle through the
images already in the skin folder; the **Import** button beside each one opens
the host file picker (`src/core/FilePicker.lua`: osascript, PowerShell,
zenity/kdialog) and copies the chosen PNG or JPG into `img/` under the name in
the SKIN field, then assigns it to that slot. Dropping a PNG or JPG on the
window does the same for whichever slot was last touched. A new bezel does not
move the screen cutout: press **Detect screen from bezel** to measure it out of
the art's alpha.
**Testing.** **Test** makes the canvas live: clicking presses real Game Boy
buttons and the footer reports what is held. **Play** saves the skin, selects
@@ -175,5 +193,3 @@ straight back into `skins/` and still opens in RetroArch.
## Not implemented
RetroArch's `analog_*`, `dpad_area`, `abxy_area` and `retrok_*` desc types.
Image assignment cycles through art already in the skin folder; there is no
file browser, so new art arrives by drag and drop.
+3 -1
View File
@@ -169,6 +169,7 @@ the NX runtime modules `src/core/NxAssetOverlay.lua`, `src/core/Platform.lua`,
`tests/engine/assets_version_fallback_test.lua`,
`tests/engine/nx_generated_guard_test.lua`,
`tests/engine/nx_yellow_boot_test.lua`,
`tests/engine/cache_fs_gold_nx_load_test.lua`,
`tests/engine/switch_diagnostics_test.lua`, `tests/engine/platform_nx_*`,
or the Switch-related workflow YAML), CI runs:
@@ -179,7 +180,8 @@ or the Switch-related workflow YAML), CI runs:
`luajit tests/switch_transfer_docs_test.lua`, and the NX engine suites
headlessly (`luajit tests/engine/assets_version_fallback_test.lua`,
`luajit tests/engine/nx_generated_guard_test.lua`,
`luajit tests/engine/nx_yellow_boot_test.lua`).
`luajit tests/engine/nx_yellow_boot_test.lua`,
`luajit tests/engine/cache_fs_gold_nx_load_test.lua`).
2. **Fused NRO build** only on the **main** repository
(`bryanthaboi/gen1recomp`), on the self-hosted Mac runner
(`scripts/build_switch.sh --fetch --fused`), and only when the workflow
+13 -9
View File
@@ -3,7 +3,7 @@
Every GitHub Release that includes Switch support ships an SD-ready zip:
`gen1recomp-*-switch.zip`. Extract it at the root of your microSD (install
or update, same steps), launch with **title override**, then import your
own legal `.gb` ROM.
own legal `.gb` / `.gbc` ROM.
> You need a console that can run Switch homebrew (custom firmware / hbmenu).
> This project does not help you set that up.
@@ -91,13 +91,14 @@ Do **not** launch from the Album applet path for normal play.
This project ships **no** game data. On first launch:
1. Put your own legally obtained Pokémon Red, Blue (`.gb`), or Yellow
(`.gbc`) dump into `switch/gen1recomp/pokemon-love2d/imports/` (the
launcher also shows the live save-dir path). All three can sit in the
1. Put your own legally obtained Pokémon Red, Blue (`.gb`), Yellow, or
Gold (`.gbc`) dump into `switch/gen1recomp/pokemon-love2d/imports/` (the
launcher also shows the live save-dir path). All four can sit in the
same folder.
2. Use **Scan again** on that game's tab (Red / Blue / Yellow). Rescan
matches by ROM SHA-1 for the open tab only. A Red dump never imports
from the Yellow tab (and vice versa).
2. Use **Scan again** on that game's tab (Red / Blue / Yellow / Gold).
Rescan matches by ROM SHA-1 for the open tab only. A Red dump never
imports from the Yellow tab (and vice versa). Gold is Beta in the
launcher; a clean US Gold dump is enough to Play.
## 5. Import / Export a raw `.sav`
@@ -109,10 +110,13 @@ SD / FTP, same transfer methods as ROMs. Paths are **per game**:
| Red | `imports/saves/red/` | `exports/red/` |
| Blue | `imports/saves/blue/` | `exports/blue/` |
| Yellow | `imports/saves/yellow/` | `exports/yellow/` |
| Gold | `imports/saves/gold/` | `exports/gold/` |
(Under the save dir `pokemon-love2d/`. The zip already creates these folders.)
(Under the save dir `pokemon-love2d/`. The zip already creates these folders.
Gold cart `.sav` import/export is not supported yet -- the folders exist so
MTP browsing matches the other games. Gold progress still saves in-engine.)
1. Copy a Gen1 `.sav` (32 KB) into that game's inbox under the save dir
1. Copy a Gen 1 `.sav` (32 KB) into that game's inbox under the save dir
([switch-transfer.md](switch-transfer.md)).
2. With the game's ROM already imported, open **that game's tab**
**SAVE FILES****Import save**. Only that folder is scanned.
+4 -4
View File
@@ -22,8 +22,8 @@ Player install (what to download, title override) stays in
| Loose iteration pair | `sdmc:/switch/gen1recomp/gen1recomp.nro` **and** `game.love` beside it |
| ROM inbox | LÖVE save dir → `imports/` (launcher shows the live `getSaveDirectory()` path; under MTP often `1: SD Card/<save identity>/imports/`) |
| Mod zip inbox | Same save dir → `imports/mods/` then MODS → **Scan again** |
| Save `.sav` inbox | Same save dir → `imports/saves/red\|blue\|yellow/` then that game's SAVE FILES → **Import save** |
| Save exports | Same save dir → `exports/red\|blue\|yellow/` (pull after **Export save**; MTP / SD / FTP) |
| Save `.sav` inbox | Same save dir → `imports/saves/red\|blue\|yellow\|gold/` then that game's SAVE FILES → **Import save** (Gold cart `.sav` not supported yet) |
| Save exports | Same save dir → `exports/red\|blue\|yellow\|gold/` (pull after **Export save**; Gold cart `.sav` not supported yet) |
| Opt-in diagnostics | Empty `switch-debug.txt` in the save dir → `switch.log` |
| Lua error log | `lua-error.log` in the save dir |
@@ -54,8 +54,8 @@ macOS, not a Mac-only requirement.
3. Create `switch/gen1recomp/` if needed; extract the release zip at SD root
(or copy NRO / `game.love` for loose).
4. For ROMs/mods/saves, open the save-dir `imports/`, `imports/mods/`,
`imports/saves/<red|blue|yellow>/`, or `exports/<red|blue|yellow>/` path the
launcher prints.
`imports/saves/<red|blue|yellow|gold>/`, or `exports/<red|blue|yellow|gold>/`
path the launcher prints.
5. Wait for the queue; refresh; exit MTP responder; title-override launch.
macOS clients often create AppleDouble sidecars (`._Something.zip`,
+65
View File
@@ -0,0 +1,65 @@
# Tiled map editing (mod authoring)
`tools/tiled_export.py` turns the imported ROM cache into a
[Tiled](https://www.mapeditor.org) workspace, so maps can be edited in a
real map editor and exported back out as a mod. The original had no map
editor at all; the port's own map data is plain Lua, which is what makes
this a data path rather than an asset path.
Editing is done in our own Tiled build,
[bryanthaboi/tiled_gen1recomp](https://github.com/bryanthaboi/tiled_gen1recomp/releases),
which ships the `gen1-mod-export` extension the workspace relies on. Grab it
from that repo's releases; upstream Tiled opens the workspace but cannot
export a mod out of it.
```sh
python3 tools/tiled_export.py # -> build/tiled/ (gitignored)
```
Then open `build/tiled/gen1.tiled-project` in that build of Tiled.
- **The overworld is one surface.** All 222 maps become `maps/*.tmj`, and
`kanto.world` places the 36 connected overworld maps at their real
connection offsets. That world is pre-loaded (seeded into the workspace's
Tiled session), so opening any one overworld map draws its neighbors around
it and you scroll and edit straight across the seams. Everything else is a
double-click away in Tiled's project panel.
- **Extending Kanto wires both ends.** A connection lives on both maps, so
hooking a new map onto a base map also emits the return connection as a
patch on that base map, keeping its other directions intact. The return
offset is derived, not guessed: all 78 vanilla reciprocal pairs satisfy
`back.offset == -offset`.
- **A Tiled tile is a gen1 block.** Each of the 24 tilesets becomes a Tiled
tileset whose tiles are its 32x32 blocks, composited from the 8x8 sheet,
so a tile layer *is* the map's `blocks` array. Warps, signs and objects
sit on the 16px cell grid in object layers, which is the grid the engine
addresses them on.
- **Collision is visible.** View > Show Tile Collision Shapes draws the real
walkability: a rectangle covers each cell whose feet tile is not in the
tileset's `walkable` list, which is the rule `src/world/Map.lua` applies.
- **Maps are shown in their real colors.** Each map is atlased in the SGB
palette it renders with, so Cerulean is blue and Lavender is purple in the
editor exactly as in game. Vanilla resolves that through a cascade with
interiors inheriting the last outdoor map, so the workspace mirrors the
cascade and walks the warp graph to colour interiors. Changing a map's
`palette` exports `palette = "..."` on the record, which beats the cascade,
and the editor offers the real palette names as a dropdown.
- **New blocks and new tilesets.** `blocksets/*.tmj` show a tileset's blocks
as raw 8x8 tiles, four by four, so new blocks can be composed there;
per-tile flags on `tilesets/tiles_*.tsj` become `walkable`, `waterTiles`,
`doorTiles` and the rest.
- **Export is a diff, not a fork of the data.** The `gen1-mod-export`
extension (shipped in `tiled_gen1recomp`) writes either one map file or a whole
loadable mod folder. An edited vanilla map diffs against the imported data
and emits `mod.content.maps:patch` carrying *only* the fields that moved, so
a mod covers the parts it changes and leaves the rest to the base game; a
new map gets `:register` at an index of 1000 or above. An unchanged map
exports nothing at all. Exports pass `tools/modkit.py validate` and `lint`.
- **Or the whole record, on request.** Ticking `exactExport` on a map switches
it to `mod.content.maps:override`, pinning the map to exactly what the
editor shows. It is off by default because an override wins outright over
any other mod patching that map, where a patch composes.
No ROM-derived art travels into an exported mod: a tileset still drawing on
the player's own imported sheet references that path rather than shipping the
pixels, and only a sheet the author supplied is copied in.
+58 -3
View File
@@ -506,6 +506,7 @@ function love.update(dt)
if TouchEditor then return TouchEditor.update(dt) end
if Studio then return Studio.update(dt) end
if Importer then return Importer:update(dt) end
if not Game then return end
-- Scripted runs (autopilot / POKEPORT_DRIVER) observe and act exactly
-- once per Game:update, so they must keep a 1:1 relationship with the
@@ -602,12 +603,14 @@ function love.keypressed(key, scancode, isrepeat)
if TouchEditor then return TouchEditor.keypressed(key) end
if Studio then return Studio.keypressed(key) end
if Importer then return Importer:keypressed(key) end
if not Game then return end
Game:keypressed(key)
end
function love.keyreleased(key)
if editorMode or TouchEditor or Studio then return end
if Importer then return end
if not Game then return end
Game:keyreleased(key)
end
@@ -625,7 +628,9 @@ function love.gamepadpressed(joystick, button)
end
return
end
if Studio then return end
if Importer then return Importer:gamepadpressed(joystick, button) end
if not Game then return end
Game:gamepadpressed(joystick, button)
end
@@ -643,7 +648,9 @@ function love.gamepadreleased(joystick, button)
end
return
end
if Studio then return end
if Importer then return Importer:gamepadreleased(joystick, button) end
if not Game then return end
Game:gamepadreleased(joystick, button)
end
@@ -661,7 +668,9 @@ function love.gamepadaxis(joystick, axis, value)
end
return
end
if Studio then return end
if Importer then return Importer:gamepadaxis(joystick, axis, value) end
if not Game then return end
Game:gamepadaxis(joystick, axis, value)
end
@@ -679,7 +688,9 @@ function love.joystickpressed(joystick, button)
end
return
end
if Studio then return end
if Importer then return Importer:joystickpressed(joystick, button) end
if not Game then return end
Game:joystickpressed(joystick, button)
end
@@ -697,7 +708,9 @@ function love.joystickreleased(joystick, button)
end
return
end
if Studio then return end
if Importer then return Importer:joystickreleased(joystick, button) end
if not Game then return end
Game:joystickreleased(joystick, button)
end
@@ -715,7 +728,9 @@ function love.joystickaxis(joystick, axis, value)
end
return
end
if Studio then return end
if Importer then return Importer:joystickaxis(joystick, axis, value) end
if not Game then return end
Game:joystickaxis(joystick, axis, value)
end
@@ -733,21 +748,25 @@ function love.joystickhat(joystick, hat, direction)
end
return
end
if Studio then return end
if Importer then return Importer:joystickhat(joystick, hat, direction) end
if not Game then return end
Game:joystickhat(joystick, hat, direction)
end
function love.joystickadded(joystick)
SwitchDiagnostics.onJoystickEvent("joystickadded", joystick)
if editorMode or TouchEditor then return end
if editorMode or TouchEditor or Studio then return end
if Importer then return end
if not Game then return end
Game:joystickadded(joystick)
end
function love.joystickremoved(joystick)
SwitchDiagnostics.onJoystickEvent("joystickremoved", joystick)
if editorMode or TouchEditor then return end
if editorMode or TouchEditor or Studio then return end
if Importer then return end
if not Game then return end
Game:joystickremoved(joystick)
end
@@ -756,29 +775,58 @@ end
-- unfocused, so reset input on either transition rather than trust it.
function love.focus(f)
if editorMode or TouchEditor then return end
if Studio then
if Studio.focus then Studio.focus(f) end
return
end
if Importer then
require("src.core.Input"):reset()
if Importer.focus then Importer:focus(f) end
return
end
if not Game then return end
Game:focus(f)
end
-- v is true when the window becomes visible again, false on minimize.
function love.visible(v)
if editorMode or TouchEditor then return end
if Studio then
if Studio.visible then Studio.visible(v) end
return
end
if Importer then
require("src.core.Input"):reset()
return
end
if not Game then return end
Game:visible(v)
end
function love.lowmemory()
if editorMode or TouchEditor or Importer then return end
if editorMode or TouchEditor or Studio or Importer then return end
if Game then Game:onResume() end
end
love.handlers = love.handlers or {}
function love.handlers.audiosuspend()
local ChipAudio = package.loaded["src.core.ChipAudio"]
if ChipAudio then pcall(ChipAudio.setSuspended, true) end
end
function love.handlers.audioreset()
local ChipAudio = package.loaded["src.core.ChipAudio"]
if ChipAudio then
pcall(ChipAudio.setSuspended, false)
pcall(ChipAudio.rebuildPlayback)
end
local Music = package.loaded["src.core.Music"]
if Music then pcall(Music.onDeviceReset) end
local Sound = package.loaded["src.core.Sound"]
if Sound then pcall(Sound.onDeviceReset) end
end
function love.touchpressed(id, x, y, dx, dy, pressure)
if editorMode then
-- iOS synthesizes mousepressed for the primary touch; forwarding here
@@ -796,12 +844,14 @@ function love.touchpressed(id, x, y, dx, dy, pressure)
if love.system.getOS() == "iOS" then return end
return TouchEditor.touchpressed(id, x, y)
end
if Studio then return end
if Importer then
-- Both mobiles: FlexLove scroll needs the real touch stream. Clicks are
-- polled inside the view; the istouch filter on mousepressed still drops
-- Android's synthesized mouse twin so Import cannot double-fire (#553).
return Importer:touchpressed(id, x, y, dx, dy, pressure)
end
if not Game then return end
Game:touchpressed(id, x, y, dx, dy, pressure)
end
@@ -811,9 +861,11 @@ function love.touchmoved(id, x, y, dx, dy, pressure)
if love.system.getOS() == "iOS" then return end
return TouchEditor.touchmoved(id, x, y)
end
if Studio then return end
if Importer then
return Importer:touchmoved(id, x, y, dx, dy, pressure)
end
if not Game then return end
Game:touchmoved(id, x, y, dx, dy, pressure)
end
@@ -823,9 +875,11 @@ function love.touchreleased(id, x, y, dx, dy, pressure)
if love.system.getOS() == "iOS" then return end
return TouchEditor.touchreleased(id, x, y)
end
if Studio then return end
if Importer then
return Importer:touchreleased(id, x, y, dx, dy, pressure)
end
if not Game then return end
Game:touchreleased(id, x, y, dx, dy, pressure)
end
@@ -837,6 +891,7 @@ function love.wheelmoved(x, y)
if TouchEditor then return end
if Studio then return Studio.wheelmoved(x, y) end
if Importer then return end
if not Game then return end
Game:wheelmoved(x, y)
end
@@ -45,6 +45,10 @@
// own, which can name a different volume on merged / adopted-SD storage.
#include "filesystem/Filesystem.h"
#include "common/Module.h"
#include "audio/Audio.h"
#include "audio/openal/Audio.h"
namespace love
{
namespace android
@@ -1320,4 +1324,70 @@ const char *love_android_poll_secondary_touch()
return event.empty() ? nullptr : event.c_str();
}
static love::audio::openal::Audio *love_android_openal_audio()
{
love::audio::Audio *audio = love::Module::getInstance<love::audio::Audio>(love::Module::M_AUDIO);
if (audio == nullptr)
return nullptr;
const char *name = audio->getName();
if (name == nullptr || strcmp(name, "love.audio.openal") != 0)
return nullptr;
return (love::audio::openal::Audio *) audio;
}
extern "C" JNIEXPORT void JNICALL
Java_org_love2d_android_GameActivity_nativeAudioFocusLost(JNIEnv *env, jclass cls)
{
(void) env;
(void) cls;
love::audio::openal::pushAudioSuspendEvent();
love::audio::openal::Audio *audio = love_android_openal_audio();
if (audio != nullptr)
audio->pauseContext();
}
extern "C" JNIEXPORT void JNICALL
Java_org_love2d_android_GameActivity_nativeAudioFocusGained(JNIEnv *env, jclass cls)
{
(void) env;
(void) cls;
love::audio::openal::Audio *audio = love_android_openal_audio();
if (audio == nullptr)
return;
audio->resumeContext();
if (!audio->isDeviceConnected())
audio->reopenDevice();
love::audio::openal::pushAudioResetEvent();
}
extern "C" JNIEXPORT void JNICALL
Java_org_love2d_android_GameActivity_nativeAudioDeviceChanged(JNIEnv *env, jclass cls)
{
(void) env;
(void) cls;
love::audio::openal::Audio *audio = love_android_openal_audio();
if (audio == nullptr)
return;
audio->pauseContext();
audio->reopenDevice();
audio->resumeContext();
love::audio::openal::pushAudioResetEvent();
}
#endif // LOVE_ANDROID
@@ -22,6 +22,7 @@
#include "common/delay.h"
#include "RecordingDevice.h"
#include "sound/Decoder.h"
#include "event/Event.h"
#include <cstdlib>
#include <iostream>
@@ -30,6 +31,10 @@
#include "common/ios.h"
#endif
#ifndef ALC_CONNECTED
#define ALC_CONNECTED 0x313
#endif
namespace love
{
namespace audio
@@ -37,9 +42,35 @@ namespace audio
namespace openal
{
Audio::PoolThread::PoolThread(Pool *pool)
: pool(pool)
static const int DISCONNECT_CHECK_INTERVAL = 200;
static void pushAudioEvent(const char *name)
{
auto eventmodule = Module::getInstance<event::Event>(Module::M_EVENT);
if (eventmodule == nullptr)
return;
event::Message *msg = new event::Message(name);
eventmodule->push(msg);
msg->release();
}
void pushAudioSuspendEvent()
{
pushAudioEvent("audiosuspend");
}
void pushAudioResetEvent()
{
pushAudioEvent("audioreset");
}
Audio::PoolThread::PoolThread(Audio *audio, Pool *pool)
: audio(audio)
, pool(pool)
, finish(false)
, paused(false)
{
threadName = "AudioPool";
}
@@ -51,6 +82,8 @@ Audio::PoolThread::~PoolThread()
void Audio::PoolThread::threadFunction()
{
int disconnectCheck = 0;
while (true)
{
{
@@ -61,7 +94,23 @@ void Audio::PoolThread::threadFunction()
}
}
if (paused.load())
{
disconnectCheck = 0;
sleep(5);
continue;
}
pool->update();
if (audio != nullptr && ++disconnectCheck >= DISCONNECT_CHECK_INTERVAL)
{
disconnectCheck = 0;
if (!audio->isDeviceConnected() && audio->reopenDevice())
pushAudioResetEvent();
}
sleep(5);
}
}
@@ -72,6 +121,11 @@ void Audio::PoolThread::setFinish()
finish = true;
}
void Audio::PoolThread::setPaused(bool paused)
{
this->paused.store(paused);
}
ALenum Audio::getFormat(int bitDepth, int channels)
{
if (bitDepth != 8 && bitDepth != 16)
@@ -99,6 +153,8 @@ Audio::Audio()
, pool(nullptr)
, poolThread(nullptr)
, distanceModel(DISTANCE_INVERSE_CLAMPED)
, alcReopenDeviceSOFT(nullptr)
, reopenChecked(false)
{
// Before opening new device, check if recording
// is requested.
@@ -189,13 +245,6 @@ Audio::Audio()
throw;
}
poolThread = new PoolThread(pool);
poolThread->start();
#ifdef LOVE_IOS
love::ios::initAudioSessionInterruptionHandler();
#endif
#ifdef LOVE_ANDROID
bool hasPauseDeviceExt = alcIsExtensionPresent(device, "ALC_SOFT_pause_device") == ALC_TRUE;
alcDevicePauseSOFT = hasPauseDeviceExt
@@ -205,6 +254,13 @@ Audio::Audio()
? (LPALCDEVICERESUMESOFT) alcGetProcAddress(device, "alcDeviceResumeSOFT")
: nullptr;
#endif
poolThread = new PoolThread(this, pool);
poolThread->start();
#ifdef LOVE_IOS
love::ios::initAudioSessionInterruptionHandler();
#endif
}
Audio::~Audio()
@@ -314,6 +370,9 @@ std::vector<love::audio::Source*> Audio::pause()
void Audio::pauseContext()
{
if (poolThread != nullptr)
poolThread->setPaused(true);
#ifdef LOVE_ANDROID
if (alcDevicePauseSOFT)
alcDevicePauseSOFT(device);
@@ -350,6 +409,52 @@ void Audio::resumeContext()
if (context && alcGetCurrentContext() != context)
alcMakeContextCurrent(context);
#endif
if (poolThread != nullptr)
poolThread->setPaused(false);
}
bool Audio::reopenDevice()
{
if (device == nullptr)
return false;
thread::Lock lock(deviceMutex);
if (!reopenChecked)
{
reopenChecked = true;
if (alcIsExtensionPresent(device, "ALC_SOFT_reopen_device") == ALC_TRUE)
alcReopenDeviceSOFT = (LPALCREOPENDEVICESOFT) alcGetProcAddress(device, "alcReopenDeviceSOFT");
}
if (alcReopenDeviceSOFT == nullptr)
return false;
alcGetError(device);
return alcReopenDeviceSOFT(device, nullptr, nullptr) == ALC_TRUE;
}
bool Audio::isDeviceConnected()
{
if (device == nullptr)
return false;
thread::Lock lock(deviceMutex);
if (alcIsExtensionPresent(device, "ALC_EXT_disconnect") != ALC_TRUE)
return true;
ALCint connected = 1;
alcGetError(device);
alcGetIntegerv(device, ALC_CONNECTED, 1, &connected);
if (alcGetError(device) != ALC_NO_ERROR)
return true;
return connected != 0;
}
void Audio::setVolume(float volume)
@@ -22,6 +22,7 @@
#define LOVE_AUDIO_OPENAL_AUDIO_H
// STD
#include <atomic>
#include <queue>
#include <map>
#include <vector>
@@ -97,6 +98,8 @@ public:
std::vector<love::audio::Source*> pause();
void pauseContext();
void resumeContext();
bool reopenDevice();
bool isDeviceConnected();
void setVolume(float volume);
float getVolume() const;
@@ -155,6 +158,7 @@ private:
class PoolThread: public thread::Threadable
{
protected:
Audio *audio;
Pool *pool;
// Set this to true when the thread should finish.
@@ -162,13 +166,16 @@ private:
// will read from it.
volatile bool finish;
std::atomic<bool> paused;
// finish lock
love::thread::MutexRef mutex;
public:
PoolThread(Pool *pool);
PoolThread(Audio *audio, Pool *pool);
virtual ~PoolThread();
void setFinish();
void setPaused(bool paused);
void threadFunction();
};
@@ -177,6 +184,13 @@ private:
DistanceModel distanceModel;
//float metersPerUnit = 1.0;
#ifndef ALC_SOFT_reopen_device
typedef ALCboolean (ALC_APIENTRY*LPALCREOPENDEVICESOFT)(ALCdevice *device, const ALCchar *deviceName, const ALCint *attribs);
#endif
LPALCREOPENDEVICESOFT alcReopenDeviceSOFT;
bool reopenChecked;
love::thread::MutexRef deviceMutex;
#ifdef LOVE_ANDROID
# ifndef ALC_SOFT_pause_device
typedef void (ALC_APIENTRY*LPALCDEVICEPAUSESOFT)(ALCdevice *device);
@@ -188,6 +202,9 @@ private:
#endif
}; // Audio
void pushAudioSuspendEvent();
void pushAudioResetEvent();
#ifdef ALC_EXT_EFX
// Effect objects
extern LPALGENEFFECTS alGenEffects;
@@ -0,0 +1,40 @@
---
name: Bug report
about: Create a report to help us improve Oboe
title: ''
labels: bug
assignees: ''
---
Android version(s):
Android device(s):
Oboe version:
App name used for testing:
(Please try to reproduce the issue using the OboeTester or an Oboe sample.)
**Short description**
(Please only report one bug per Issue. Do not combine multiple bugs.)
**Steps to reproduce**
**Expected behavior**
**Actual behavior**
**Device**
Please list which devices have this bug.
If device specific, and you are on Linux or a Macintosh, connect the device and please share the result for the following script. This gets properties of the device.
```
for p in \
ro.product.brand ro.product.manufacturer ro.product.model \
ro.product.device ro.product.cpu.abi ro.build.description \
ro.hardware ro.hardware.chipname ro.arch "| grep aaudio";
do echo "$p = $(adb shell getprop $p)"; done
```
**Any additional context**
If applicable, please attach a few seconds of an uncompressed recording of the sound in a WAV or AIFF file.
+36
View File
@@ -0,0 +1,36 @@
# To get started with Dependabot version updates, you'll need to specify which
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://docs.github.com/github/administering-a-repository/configuration-options-for-dependency-updates
version: 2
#Workaround for https://github.com/dependabot/dependabot-core/issues/6888#issuecomment-1539501116
registries:
maven-google:
type: maven-repository
url: "https://dl.google.com/dl/android/maven2/"
updates:
#Check for updates to Github Actions
- package-ecosystem: "github-actions"
directory: "/" #Location of package manifests
target-branch: "main"
open-pull-requests-limit: 5
labels:
- "dependencies"
- "dependencies/github-actions"
schedule:
interval: "daily"
#Check updates for Gradle dependencies
- package-ecosystem: "gradle"
registries:
- maven-google
directory: "/" #Location of package manifests
target-branch: "main"
open-pull-requests-limit: 10
labels:
- "dependencies"
- "dependencies/gradle"
schedule:
interval: "daily"
@@ -0,0 +1,38 @@
name: Build CI
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: set up JDK 17
uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: 17
- name: build samples and apps
uses: github/codeql-action/init@v3
with:
languages: cpp
- run: |
pushd samples
chmod +x gradlew
./gradlew -q clean bundleDebug
popd
pushd apps/OboeTester
chmod +x gradlew
./gradlew -q clean bundleDebug
popd
pushd apps/fxlab
chmod +x gradlew
./gradlew -q clean bundleDebug
popd
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v3
@@ -0,0 +1,24 @@
name: Update Docs
on:
push:
branches: [ main ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Doxygen Action
uses: mattnotmitt/doxygen-action@v1.9.8
with:
doxyfile-path: "./Doxyfile"
working-directory: "."
- name: Deploy
uses: peaceiris/actions-gh-pages@v4
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: ./docs/reference
@@ -4,3 +4,5 @@
.cxx/
.idea
build
.logpile
+14 -17
View File
@@ -1,32 +1,21 @@
LOCAL_PATH:= $(call my-dir)
include $(CLEAR_VARS)
#
# Module
#
LOCAL_MODULE := oboe
LOCAL_ARM_NEON := true
#
# Flags
#
LOCAL_CFLAGS := -Wall -Wextra-semi -Wshadow -Wshadow-field
LOCAL_CPPFLAGS := -std=c++14
LOCAL_CPPFLAGS := -std=c++17
#
# Include paths
#
LOCAL_C_INCLUDES := \
$(LOCAL_PATH)/include \
$(LOCAL_PATH)/src
LOCAL_EXPORT_C_INCLUDES := $(LOCAL_PATH)/include
#
# Source files
#
LOCAL_SRC_FILES := \
src/aaudio/AAudioLoader.cpp \
src/aaudio/AudioStreamAAudio.cpp \
src/common/AdpfWrapper.cpp \
src/common/AudioSourceCaller.cpp \
src/common/AudioStream.cpp \
src/common/AudioStreamBuilder.cpp \
@@ -36,8 +25,11 @@ LOCAL_SRC_FILES := \
src/common/FixedBlockReader.cpp \
src/common/FixedBlockWriter.cpp \
src/common/LatencyTuner.cpp \
src/common/OboeExtensions.cpp \
src/common/SourceFloatCaller.cpp \
src/common/SourceI16Caller.cpp \
src/common/SourceI24Caller.cpp \
src/common/SourceI32Caller.cpp \
src/common/Utilities.cpp \
src/common/QuirksManager.cpp \
src/fifo/FifoBuffer.cpp \
@@ -45,17 +37,26 @@ LOCAL_SRC_FILES := \
src/fifo/FifoControllerBase.cpp \
src/fifo/FifoControllerIndirect.cpp \
src/flowgraph/FlowGraphNode.cpp \
src/flowgraph/ChannelCountConverter.cpp \
src/flowgraph/ClipToRange.cpp \
src/flowgraph/Limiter.cpp \
src/flowgraph/ManyToMultiConverter.cpp \
src/flowgraph/MonoBlend.cpp \
src/flowgraph/MonoToMultiConverter.cpp \
src/flowgraph/MultiToManyConverter.cpp \
src/flowgraph/MultiToMonoConverter.cpp \
src/flowgraph/RampLinear.cpp \
src/flowgraph/SampleRateConverter.cpp \
src/flowgraph/SinkFloat.cpp \
src/flowgraph/SinkI16.cpp \
src/flowgraph/SinkI24.cpp \
src/flowgraph/SinkI32.cpp \
src/flowgraph/SinkI8_24.cpp \
src/flowgraph/SourceFloat.cpp \
src/flowgraph/SourceI16.cpp \
src/flowgraph/SourceI24.cpp \
src/flowgraph/SourceI32.cpp \
src/flowgraph/SourceI8_24.cpp \
src/flowgraph/resampler/IntegerRatio.cpp \
src/flowgraph/resampler/LinearResampler.cpp \
src/flowgraph/resampler/MultiChannelResampler.cpp \
@@ -75,10 +76,6 @@ LOCAL_SRC_FILES := \
src/common/Trace.cpp \
src/common/Version.cpp
#
# Libraries related
#
LOCAL_LDLIBS := -llog
# Build
include $(BUILD_STATIC_LIBRARY)
@@ -9,6 +9,7 @@ project(oboe)
set (oboe_sources
src/aaudio/AAudioLoader.cpp
src/aaudio/AudioStreamAAudio.cpp
src/common/AdpfWrapper.cpp
src/common/AudioSourceCaller.cpp
src/common/AudioStream.cpp
src/common/AudioStreamBuilder.cpp
@@ -18,26 +19,38 @@ set (oboe_sources
src/common/FixedBlockReader.cpp
src/common/FixedBlockWriter.cpp
src/common/LatencyTuner.cpp
src/common/OboeExtensions.cpp
src/common/SourceFloatCaller.cpp
src/common/SourceI16Caller.cpp
src/common/SourceI24Caller.cpp
src/common/SourceI32Caller.cpp
src/common/Utilities.cpp
src/common/QuirksManager.cpp
src/fifo/FifoBuffer.cpp
src/fifo/FifoController.cpp
src/fifo/FifoControllerBase.cpp
src/fifo/FifoControllerIndirect.cpp
src/flowgraph/FlowGraphNode.cpp
src/flowgraph/FlowGraphNode.cpp
src/flowgraph/ChannelCountConverter.cpp
src/flowgraph/ClipToRange.cpp
src/flowgraph/Limiter.cpp
src/flowgraph/ManyToMultiConverter.cpp
src/flowgraph/MonoBlend.cpp
src/flowgraph/MonoToMultiConverter.cpp
src/flowgraph/MultiToManyConverter.cpp
src/flowgraph/MultiToMonoConverter.cpp
src/flowgraph/RampLinear.cpp
src/flowgraph/SampleRateConverter.cpp
src/flowgraph/SinkFloat.cpp
src/flowgraph/SinkI16.cpp
src/flowgraph/SinkI24.cpp
src/flowgraph/SinkI32.cpp
src/flowgraph/SinkI8_24.cpp
src/flowgraph/SourceFloat.cpp
src/flowgraph/SourceI16.cpp
src/flowgraph/SourceI24.cpp
src/flowgraph/SourceI32.cpp
src/flowgraph/SourceI8_24.cpp
src/flowgraph/resampler/IntegerRatio.cpp
src/flowgraph/resampler/LinearResampler.cpp
src/flowgraph/resampler/MultiChannelResampler.cpp
@@ -70,18 +83,23 @@ target_include_directories(oboe
# Enable -Ofast
target_compile_options(oboe
PRIVATE
-std=c++14
-std=c++17
-Wall
-Wextra-semi
-Wshadow
-Wshadow-field
-Ofast
"$<$<CONFIG:RELEASE>:-Ofast>"
"$<$<CONFIG:DEBUG>:-O3>"
"$<$<CONFIG:DEBUG>:-Werror>")
# Enable logging of D,V for debug builds
target_compile_definitions(oboe PUBLIC $<$<CONFIG:DEBUG>:OBOE_ENABLE_LOGGING=1>)
option(OBOE_DO_NOT_DEFINE_OPENSL_ES_CONSTANTS "Do not define OpenSLES constants" OFF)
target_compile_definitions(oboe PRIVATE $<$<BOOL:${OBOE_DO_NOT_DEFINE_OPENSL_ES_CONSTANTS}>:DO_NOT_DEFINE_OPENSL_ES_CONSTANTS=1>)
target_link_libraries(oboe PRIVATE log OpenSLES)
target_link_options(oboe PRIVATE "-Wl,-z,max-page-size=16384")
# When installing oboe put the libraries in the lib/<ABI> folder e.g. lib/arm64-v8a
install(TARGETS oboe
@@ -89,4 +107,4 @@ install(TARGETS oboe
ARCHIVE DESTINATION lib/${ANDROID_ABI})
# Also install the headers
install(DIRECTORY include/oboe DESTINATION include)
install(DIRECTORY include/oboe DESTINATION include)
@@ -1 +0,0 @@
Please see the CONTRIBUTING.md file for more information.
+2 -2
View File
@@ -38,7 +38,7 @@ PROJECT_NAME = "Oboe"
# could be handy for archiving the generated documentation or if some version
# control system is used.
PROJECT_NUMBER = 1.2
PROJECT_NUMBER =
# Using the PROJECT_BRIEF tag one can provide an optional one line description
# for a project that appears at the top of each page and should give viewer a
@@ -58,7 +58,7 @@ PROJECT_LOGO =
# entered, it will be relative to the location where doxygen was started. If
# left blank the current directory will be used.
OUTPUT_DIRECTORY = docs
OUTPUT_DIRECTORY = ./docs
# If the CREATE_SUBDIRS tag is set to YES then doxygen will create 4096 sub-
# directories (in 2 levels) under the output directory of each output format and
-1
View File
@@ -1 +0,0 @@
Please see the README.md file for more information.
+14 -13
View File
@@ -1,4 +1,4 @@
# Oboe [![Build Status](https://travis-ci.org/google/oboe.svg?branch=master)](https://travis-ci.org/google/oboe)
# Oboe [![Build CI](https://github.com/google/oboe/workflows/Build%20CI/badge.svg)](https://github.com/google/oboe/actions)
[![Introduction to Oboe video](docs/images/getting-started-video.jpg)](https://www.youtube.com/watch?v=csfHAbr5ilI&list=PLWz5rJ2EKKc_duWv9IPNvx9YBudNMmLSa)
@@ -9,35 +9,36 @@ Oboe is a C++ library which makes it easy to build high-performance audio apps o
- Chooses the audio API (OpenSL ES on API 16+ or AAudio on API 27+) which will give the best audio performance on the target Android device
- Automatic latency tuning
- Modern C++ allowing you to write clean, elegant code
- [Used by popular apps and frameworks](docs/AppsUsingOboe.md)
- Workarounds for some known issues
- [Used by popular apps and frameworks](https://github.com/google/oboe/wiki/AppsUsingOboe)
## Requirements
To build Oboe you'll need a compiler which supports C++14 and the Android header files. The easiest way to obtain these is by downloading the Android NDK r17 or above. It can be installed using Android Studio's SDK manager, or via [direct download](https://developer.android.com/ndk/downloads/).
## API Documentation
## Documentation
- [Getting Started Guide](docs/GettingStarted.md)
- [Full Guide to Oboe](docs/FullGuide.md)
- [API reference](https://google.github.io/oboe/reference)
- [Tech Notes](docs/notes/)
- [API reference](https://google.github.io/oboe)
- [History of Audio features/bugs by Android version](docs/AndroidAudioHistory.md)
- [Migration guide for apps using OpenSL ES](docs/OpenSLESMigration.md)
- [Frequently Asked Questions](docs/FAQ.md) (FAQ)
- [Wiki](https://github.com/google/oboe/wiki)
- [Our roadmap](https://github.com/google/oboe/milestones) - Vote on a feature/issue by adding a thumbs up to the first comment.
### Community
- Reddit: [r/androidaudiodev](https://www.reddit.com/r/androidaudiodev/)
- StackOverflow: [#oboe](https://stackoverflow.com/questions/tagged/oboe)
## Testing
- [**OboeTester** app for measuring latency, glitches, etc.](https://github.com/google/oboe/tree/master/apps/OboeTester/docs)
- [Oboe unit tests](https://github.com/google/oboe/tree/master/tests)
- [**OboeTester** app for measuring latency, glitches, etc.](apps/OboeTester/docs)
- [Oboe unit tests](tests)
## Videos
- [Getting started with Oboe](https://www.youtube.com/playlist?list=PLWz5rJ2EKKc_duWv9IPNvx9YBudNMmLSa)
- [Low Latency Audio - Because Your Ears Are Worth It](https://www.youtube.com/watch?v=8vOf_fDtur4) (Android Dev Summit '18)
- [Real-time audio with the 100 oscillator synthesizer](https://www.youtube.com/watch?v=J04iPJBkAKs) (DroidCon Berlin '18)
- [Winning on Android](https://www.youtube.com/watch?v=tWBojmBpS74) - How to optimize an Android audio app. (ADC '18)
- [Real-Time Processing on Android](https://youtu.be/hY9BrS2uX-c) (ADC '19)
## Sample code and apps
- Sample apps can be found in the [samples directory](samples).
- A complete "effects processor" app called FXLab can be found in the [apps/fxlab folder](apps/fxlab).
- Also check out the [Rhythm Game codelab](https://codelabs.developers.google.com/codelabs/musicalgame-using-oboe/index.html#0).
- Also check out the [Rhythm Game codelab](https://developer.android.com/codelabs/musicalgame-using-oboe?hl=en#0).
### Third party sample code
- [Ableton Link integration demo](https://github.com/jbloit/AndroidLinkAudio) (author: jbloit)
@@ -6,6 +6,8 @@
/build/
.idea/
/app/build/
/app/release/
/app/debug/
/app/app.iml
*.iml
/app/externalNativeBuild/
@@ -1,11 +1,14 @@
cmake_minimum_required(VERSION 3.4.1)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Werror -Wall -std=c++14")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Werror -Wall -std=c++17 -fvisibility=hidden")
set(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -O2")
set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} -O3")
link_directories(${CMAKE_CURRENT_LIST_DIR}/..)
# Increment this number when adding files to OboeTester => 105
# The change in this file will help Android Studio resync
# and generate new build files that reference the new code.
file(GLOB_RECURSE app_native_sources src/main/cpp/*)
### Name must match loadLibrary() call in MainActivity.java
@@ -30,5 +33,4 @@ include_directories(
# link to oboe
target_link_libraries(oboetester log oboe atomic)
# bump 2 to resync CMake
target_link_options(oboetester PRIVATE "-Wl,-z,max-page-size=16384")
@@ -1,18 +1,17 @@
apply plugin: 'com.android.application'
android {
compileSdkVersion 28
compileSdkVersion 34
defaultConfig {
applicationId = "com.google.sample.oboe.manualtest"
applicationId = "com.mobileer.oboetester"
minSdkVersion 23
targetSdkVersion 28
// Also update the version in the AndroidManifest.xml file.
versionCode 32
versionName "1.5.24"
targetSdkVersion 34
versionCode 91
versionName "2.7.2"
testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
externalNativeBuild {
cmake {
cppFlags "-std=c++14"
cppFlags "-std=c++17"
abiFilters "x86", "x86_64", "armeabi-v7a", "arm64-v8a"
}
}
@@ -31,14 +30,15 @@ android {
path "CMakeLists.txt"
}
}
namespace 'com.mobileer.oboetester'
}
dependencies {
implementation fileTree(include: ['*.jar'], dir: 'libs')
implementation 'com.android.support.constraint:constraint-layout:2.0.0-beta4'
implementation "androidx.core:core-ktx:1.9.0"
implementation 'androidx.constraintlayout:constraintlayout:2.1.4'
implementation 'androidx.appcompat:appcompat:1.6.1'
testImplementation 'junit:junit:4.13-beta-3'
implementation 'com.android.support:appcompat-v7:28.0.0'
androidTestImplementation 'com.android.support.test:runner:1.0.2'
androidTestImplementation 'com.android.support.test.espresso:espresso-core:3.0.2'
}
androidTestImplementation 'androidx.test.ext:junit:1.1.5'
androidTestImplementation 'androidx.test.espresso:espresso-core:3.5.1'
}
@@ -1,100 +1,138 @@
<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
package="com.google.sample.oboe.manualtest"
android:versionCode="32"
android:versionName="1.5.24">
<!-- versionCode and versionName also have to be updated in build.gradle -->
<uses-feature android:name="android.hardware.microphone" android:required="true" />
<uses-feature android:name="android.hardware.audio.output" android:required="true" />
<uses-feature android:name="android.software.midi" android:required="true" />
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
<uses-feature
android:name="android.hardware.microphone"
android:required="false" />
<uses-feature
android:name="android.hardware.audio.output"
android:required="true" />
<uses-feature
android:name="android.hardware.touchscreen"
android:required="false" />
<uses-feature
android:name="android.software.midi"
android:required="false" />
<uses-feature
android:name="android.software.leanback"
android:required="false" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS" />
<uses-permission android:name="android.permission.INTERNET" />
<!-- debug-writing file need external storage writing -->
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.READ_PHONE_STATE" />
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_MEDIA_PLAYBACK" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_MICROPHONE" />
<application
android:allowBackup="false"
android:fullBackupContent="false"
android:icon="@mipmap/ic_launcher"
android:label="@string/app_name"
android:supportsRtl="true"
android:theme="@style/AppTheme">
android:theme="@style/AppTheme"
android:requestLegacyExternalStorage="true"
android:banner="@mipmap/ic_launcher">
<activity
android:name="com.google.sample.oboe.manualtest.MainActivity"
android:name=".MainActivity"
android:launchMode="singleTask"
android:label="@string/app_name"
android:screenOrientation="portrait">
android:screenOrientation="portrait"
android:exported="true">
<intent-filter>
<action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" />
<category android:name="android.intent.category.LEANBACK_LAUNCHER" />
</intent-filter>
</activity>
<activity
android:name="com.google.sample.oboe.manualtest.TestOutputActivity"
android:name=".TestOutputActivity"
android:label="@string/title_activity_test_output"
android:screenOrientation="portrait">
</activity>
android:screenOrientation="portrait" />
<activity
android:name="com.google.sample.oboe.manualtest.TestInputActivity"
android:name=".TestInputActivity"
android:label="@string/title_activity_test_input"
android:screenOrientation="portrait">
</activity>
android:screenOrientation="portrait" />
<activity
android:name="com.google.sample.oboe.manualtest.TapToToneActivity"
android:name=".TapToToneActivity"
android:label="@string/title_activity_output_latency"
android:screenOrientation="portrait">
</activity>
android:screenOrientation="portrait" />
<activity
android:name="com.google.sample.oboe.manualtest.RecorderActivity"
android:name=".RecorderActivity"
android:label="@string/title_activity_recorder"
android:screenOrientation="portrait">
</activity>
android:screenOrientation="portrait" />
<activity
android:name="com.google.sample.oboe.manualtest.EchoActivity"
android:name=".EchoActivity"
android:label="@string/title_activity_echo"
android:screenOrientation="portrait">
</activity>
android:screenOrientation="portrait" />
<activity
android:name="com.google.sample.oboe.manualtest.RoundTripLatencyActivity"
android:name=".RoundTripLatencyActivity"
android:label="@string/title_activity_rt_latency"
android:screenOrientation="portrait">
</activity>
android:screenOrientation="portrait" />
<activity
android:name="com.google.sample.oboe.manualtest.ManualGlitchActivity"
android:name=".ManualGlitchActivity"
android:label="@string/title_activity_glitches"
android:screenOrientation="portrait">
</activity>
android:screenOrientation="portrait" />
<activity
android:name="com.google.sample.oboe.manualtest.AutoGlitchActivity"
android:label="@string/title_activity_glitches"
android:screenOrientation="portrait">
</activity>
android:name=".AutomatedGlitchActivity"
android:label="@string/title_activity_auto_glitches"
android:screenOrientation="portrait" />
<activity
android:name="com.google.sample.oboe.manualtest.TestDisconnectActivity"
android:name=".TestDisconnectActivity"
android:label="@string/title_test_disconnect"
android:screenOrientation="portrait">
</activity>
android:screenOrientation="portrait" />
<activity
android:name=".DeviceReportActivity"
android:label="@string/title_report_devices"
android:screenOrientation="portrait" />
<activity
android:name=".TestDataPathsActivity"
android:label="@string/title_data_paths"
android:screenOrientation="portrait" />
<activity
android:name=".ExtraTestsActivity"
android:exported="true"
android:label="@string/title_extra_tests"
android:screenOrientation="portrait" />
<activity
android:name=".ExternalTapToToneActivity"
android:label="@string/title_external_tap"
android:exported="true"
android:screenOrientation="portrait" />
<activity
android:name=".TestPlugLatencyActivity"
android:label="@string/title_plug_latency"
android:exported="true"
android:screenOrientation="portrait" />
<activity
android:name=".TestErrorCallbackActivity"
android:label="@string/title_error_callback"
android:exported="true"
android:screenOrientation="portrait" />
<activity
android:name=".TestRouteDuringCallbackActivity"
android:label="@string/title_route_during_callback"
android:exported="true"
android:screenOrientation="portrait" />
<activity
android:name=".DynamicWorkloadActivity"
android:label="@string/title_dynamic_load"
android:exported="true"
android:screenOrientation="portrait" />
<activity
android:name=".TestColdStartLatencyActivity"
android:label="@string/title_cold_start_latency"
android:exported="true"
android:screenOrientation="portrait" />
<activity
android:name=".TestRapidCycleActivity"
android:label="@string/title_rapid_cycle"
android:exported="true"
android:screenOrientation="portrait" />
<service
android:name="com.google.sample.oboe.manualtest.AudioMidiTester"
android:permission="android.permission.BIND_MIDI_DEVICE_SERVICE">
android:name=".MidiTapTester"
android:permission="android.permission.BIND_MIDI_DEVICE_SERVICE"
android:exported="true">
<intent-filter>
<action android:name="android.media.midi.MidiDeviceService" />
</intent-filter>
@@ -104,16 +142,21 @@
android:resource="@xml/service_device_info" />
</service>
<service
android:name=".AudioForegroundService"
android:foregroundServiceType="mediaPlayback|microphone"
android:exported="false">
</service>
<provider
android:name="android.support.v4.content.FileProvider"
android:name="androidx.core.content.FileProvider"
android:authorities="${applicationId}.provider"
android:exported="false"
android:grantUriPermissions="true">
<meta-data
android:name="android.support.FILE_PROVIDER_PATHS"
android:resource="@xml/provider_paths"/>
android:resource="@xml/provider_paths" />
</provider>
</application>
</manifest>
</manifest>
@@ -17,29 +17,24 @@
#include <cstring>
#include <sched.h>
#include "common/OboeDebug.h"
#include "oboe/Oboe.h"
#include "AudioStreamGateway.h"
using namespace flowgraph;
using namespace oboe::flowgraph;
oboe::DataCallbackResult AudioStreamGateway::onAudioReady(
oboe::AudioStream *audioStream,
void *audioData,
int numFrames) {
if (!mSchedulerChecked) {
mScheduler = sched_getscheduler(gettid());
mSchedulerChecked = true;
}
maybeHang(getNanoseconds());
printScheduler();
if (mAudioSink != nullptr) {
mAudioSink->read(mFramePosition, audioData, numFrames);
mFramePosition += numFrames;
mAudioSink->read(audioData, numFrames);
}
return oboe::DataCallbackResult::Continue;
}
int AudioStreamGateway::getScheduler() {
return mScheduler;
}
@@ -21,24 +21,21 @@
#include "flowgraph/FlowGraphNode.h"
#include "oboe/Oboe.h"
#include "OboeTesterStreamCallback.h"
using namespace flowgraph;
using namespace oboe::flowgraph;
/**
* Bridge between an audio flowgraph and an audio device.
* Pass in an AudioSink and then pass
* this object to the AudioStreamBuilder as a callback.
*/
class AudioStreamGateway : public oboe::AudioStreamCallback {
class AudioStreamGateway : public OboeTesterStreamCallback {
public:
// AudioStreamGateway(int samplesPerFrame);
virtual ~AudioStreamGateway() = default;
void setAudioSink(std::shared_ptr<flowgraph::FlowGraphSink> sink) {
void setAudioSink(std::shared_ptr<oboe::flowgraph::FlowGraphSink> sink) {
mAudioSink = sink;
if (sink) {
mFramePosition = sink->getLastFramePosition();
}
}
/**
@@ -49,13 +46,9 @@ public:
void *audioData,
int numFrames) override;
int getScheduler();
private:
int64_t mFramePosition = 0;
bool mSchedulerChecked = false;
int mScheduler;
std::shared_ptr<flowgraph::FlowGraphSink> mAudioSink;
std::shared_ptr<oboe::flowgraph::FlowGraphSink> mAudioSink;
};
@@ -0,0 +1,91 @@
/*
* Copyright 2021 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#include "FormatConverterBox.h"
FormatConverterBox::FormatConverterBox(int32_t maxSamples,
oboe::AudioFormat inputFormat,
oboe::AudioFormat outputFormat) {
mInputFormat = inputFormat;
mOutputFormat = outputFormat;
mMaxSamples = maxSamples;
mInputBuffer = std::make_unique<uint8_t[]>(maxSamples * sizeof(int32_t));
mOutputBuffer = std::make_unique<uint8_t[]>(maxSamples * sizeof(int32_t));
mSource.reset();
switch (mInputFormat) {
case oboe::AudioFormat::I16:
case oboe::AudioFormat::IEC61937:
mSource = std::make_unique<oboe::flowgraph::SourceI16>(1);
break;
case oboe::AudioFormat::I24:
mSource = std::make_unique<oboe::flowgraph::SourceI24>(1);
break;
case oboe::AudioFormat::I32:
mSource = std::make_unique<oboe::flowgraph::SourceI32>(1);
break;
case oboe::AudioFormat::Float:
case oboe::AudioFormat::Invalid:
case oboe::AudioFormat::Unspecified:
mSource = std::make_unique<oboe::flowgraph::SourceFloat>(1);
break;
}
mSink.reset();
switch (mOutputFormat) {
case oboe::AudioFormat::I16:
case oboe::AudioFormat::IEC61937:
mSink = std::make_unique<oboe::flowgraph::SinkI16>(1);
break;
case oboe::AudioFormat::I24:
mSink = std::make_unique<oboe::flowgraph::SinkI24>(1);
break;
case oboe::AudioFormat::I32:
mSink = std::make_unique<oboe::flowgraph::SinkI32>(1);
break;
case oboe::AudioFormat::Float:
case oboe::AudioFormat::Invalid:
case oboe::AudioFormat::Unspecified:
mSink = std::make_unique<oboe::flowgraph::SinkFloat>(1);
break;
}
if (mSource && mSink) {
mSource->output.connect(&mSink->input);
mSink->pullReset();
}
}
int32_t FormatConverterBox::convertInternalBuffers(int32_t numSamples) {
assert(numSamples <= mMaxSamples);
return convert(getOutputBuffer(), numSamples, getInputBuffer());
}
int32_t FormatConverterBox::convertToInternalOutput(int32_t numSamples, const void *inputBuffer) {
assert(numSamples <= mMaxSamples);
return convert(getOutputBuffer(), numSamples, inputBuffer);
}
int32_t FormatConverterBox::convertFromInternalInput(void *outputBuffer, int32_t numSamples) {
assert(numSamples <= mMaxSamples);
return convert(outputBuffer, numSamples, getInputBuffer());
}
int32_t FormatConverterBox::convert(void *outputBuffer, int32_t numSamples, const void *inputBuffer) {
mSource->setData(inputBuffer, numSamples);
return mSink->read(outputBuffer, numSamples);
}
@@ -0,0 +1,102 @@
/*
* Copyright 2021 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#ifndef OBOETESTER_FORMAT_CONVERTER_BOX_H
#define OBOETESTER_FORMAT_CONVERTER_BOX_H
#include <unistd.h>
#include <sys/types.h>
#include "oboe/Oboe.h"
#include "flowgraph/SinkFloat.h"
#include "flowgraph/SinkI16.h"
#include "flowgraph/SinkI24.h"
#include "flowgraph/SinkI32.h"
#include "flowgraph/SourceFloat.h"
#include "flowgraph/SourceI16.h"
#include "flowgraph/SourceI24.h"
#include "flowgraph/SourceI32.h"
/**
* Use flowgraph modules to convert between the various data formats.
*
* Note that this does not do channel conversions.
*/
class FormatConverterBox {
public:
FormatConverterBox(int32_t maxSamples,
oboe::AudioFormat inputFormat,
oboe::AudioFormat outputFormat);
/**
* @return internal buffer used to store input data
*/
void *getOutputBuffer() {
return (void *) mOutputBuffer.get();
};
/**
* @return internal buffer used to store output data
*/
void *getInputBuffer() {
return (void *) mInputBuffer.get();
};
/** Convert the data from inputFormat to outputFormat
* using both internal buffers.
*/
int32_t convertInternalBuffers(int32_t numSamples);
/**
* Convert data from external buffer into internal output buffer.
* @param numSamples
* @param inputBuffer
* @return
*/
int32_t convertToInternalOutput(int32_t numSamples, const void *inputBuffer);
/**
*
* Convert data from internal input buffer into external output buffer.
* @param outputBuffer
* @param numSamples
* @return
*/
int32_t convertFromInternalInput(void *outputBuffer, int32_t numSamples);
/**
* Convert data formats between the specified external buffers.
* @param outputBuffer
* @param numSamples
* @param inputBuffer
* @return
*/
int32_t convert(void *outputBuffer, int32_t numSamples, const void *inputBuffer);
private:
oboe::AudioFormat mInputFormat{oboe::AudioFormat::Invalid};
oboe::AudioFormat mOutputFormat{oboe::AudioFormat::Invalid};
int32_t mMaxSamples = 0;
std::unique_ptr<uint8_t[]> mInputBuffer;
std::unique_ptr<uint8_t[]> mOutputBuffer;
std::unique_ptr<oboe::flowgraph::FlowGraphSourceBuffered> mSource;
std::unique_ptr<oboe::flowgraph::FlowGraphSink> mSink;
};
#endif //OBOETESTER_FORMAT_CONVERTER_BOX_H
@@ -19,28 +19,39 @@
oboe::Result FullDuplexAnalyzer::start() {
getLoopbackProcessor()->setSampleRate(getOutputStream()->getSampleRate());
getLoopbackProcessor()->onStartTest();
return FullDuplexStream::start();
getLoopbackProcessor()->prepareToTest();
mWriteReadDeltaValid = false;
return FullDuplexStreamWithConversion::start();
}
oboe::DataCallbackResult FullDuplexAnalyzer::onBothStreamsReady(
const void *inputData,
oboe::DataCallbackResult FullDuplexAnalyzer::onBothStreamsReadyFloat(
const float *inputData,
int numInputFrames,
void *outputData,
float *outputData,
int numOutputFrames) {
int32_t inputStride = getInputStream()->getChannelCount();
int32_t outputStride = getOutputStream()->getChannelCount();
float *inputFloat = (float *) inputData;
float *outputFloat = (float *) outputData;
auto *inputFloat = static_cast<const float *>(inputData);
float *outputFloat = outputData;
// Get atomic snapshot of the relative frame positions so they
// can be used to calculate timestamp latency.
int64_t framesRead = getInputStream()->getFramesRead();
int64_t framesWritten = getOutputStream()->getFramesWritten();
mWriteReadDelta = framesWritten - framesRead;
mWriteReadDeltaValid = true;
(void) getLoopbackProcessor()->process(inputFloat, inputStride, numInputFrames,
outputFloat, outputStride, numOutputFrames);
// write the first channel of output and input to the stereo recorder
// Save data for later analysis or for writing to a WAVE file.
if (mRecording != nullptr) {
float buffer[2];
int numBoth = std::min(numInputFrames, numOutputFrames);
// Offset to the selected channels that we are analyzing.
inputFloat += getLoopbackProcessor()->getInputChannel();
outputFloat += getLoopbackProcessor()->getOutputChannel();
for (int i = 0; i < numBoth; i++) {
buffer[0] = *outputFloat;
outputFloat += outputStride;
@@ -48,14 +59,15 @@ oboe::DataCallbackResult FullDuplexAnalyzer::onBothStreamsReady(
inputFloat += inputStride;
mRecording->write(buffer, 1);
}
// Handle mismatch in in numFrames.
buffer[0] = 0.0f; // gap in output
// Handle mismatch in numFrames.
const float gapMarker = -0.9f; // Recognizable value so we can tell underruns from DSP gaps.
buffer[0] = gapMarker; // gap in output
for (int i = numBoth; i < numInputFrames; i++) {
buffer[1] = *inputFloat;
inputFloat += inputStride;
mRecording->write(buffer, 1);
}
buffer[1] = 0.0f; // gap in input
buffer[1] = gapMarker; // gap in input
for (int i = numBoth; i < numOutputFrames; i++) {
buffer[0] = *outputFloat;
outputFloat += outputStride;
@@ -21,40 +21,52 @@
#include <sys/types.h>
#include "oboe/Oboe.h"
#include "FullDuplexStream.h"
#include "analyzer/LatencyAnalyzer.h"
#include "FullDuplexStreamWithConversion.h"
#include "MultiChannelRecording.h"
class FullDuplexAnalyzer : public FullDuplexStream {
class FullDuplexAnalyzer : public FullDuplexStreamWithConversion {
public:
FullDuplexAnalyzer() {}
FullDuplexAnalyzer(LoopbackProcessor *processor)
: mLoopbackProcessor(processor) {
}
/**
* Called when data is available on both streams.
* Caller should override this method.
*/
oboe::DataCallbackResult onBothStreamsReady(
const void *inputData,
oboe::DataCallbackResult onBothStreamsReadyFloat(
const float *inputData,
int numInputFrames,
void *outputData,
float *outputData,
int numOutputFrames
) override;
oboe::Result start() override;
bool isDone() {
return false;
LoopbackProcessor *getLoopbackProcessor() {
return mLoopbackProcessor;
}
virtual LoopbackProcessor *getLoopbackProcessor() = 0;
void setRecording(MultiChannelRecording *recording) {
mRecording = recording;
}
bool isWriteReadDeltaValid() {
return mWriteReadDeltaValid;
}
int64_t getWriteReadDelta() {
return mWriteReadDelta;
}
private:
MultiChannelRecording *mRecording = nullptr;
LoopbackProcessor * const mLoopbackProcessor;
std::atomic<bool> mWriteReadDeltaValid{false};
std::atomic<int64_t> mWriteReadDelta{0};
};
@@ -20,28 +20,46 @@
oboe::Result FullDuplexEcho::start() {
int32_t delayFrames = (int32_t) (kMaxDelayTimeSeconds * getOutputStream()->getSampleRate());
mDelayLine = std::make_unique<InterpolatingDelayLine>(delayFrames);
return FullDuplexStream::start();
// Use peak detector for input streams
mNumChannels = getInputStream()->getChannelCount();
mPeakDetectors = std::make_unique<PeakDetector[]>(mNumChannels);
return FullDuplexStreamWithConversion::start();
}
oboe::DataCallbackResult FullDuplexEcho::onBothStreamsReady(
const void *inputData,
double FullDuplexEcho::getPeakLevel(int index) {
if (mPeakDetectors == nullptr) {
LOGE("%s() called before setup()", __func__);
return -1.0;
} else if (index < 0 || index >= mNumChannels) {
LOGE("%s(), index out of range, 0 <= %d < %d", __func__, index, mNumChannels.load());
return -2.0;
}
return mPeakDetectors[index].getLevel();
}
oboe::DataCallbackResult FullDuplexEcho::onBothStreamsReadyFloat(
const float *inputData,
int numInputFrames,
void *outputData,
float *outputData,
int numOutputFrames) {
// FIXME only handles matching stream formats.
// TODO Add delay node
// TODO use flowgraph to handle format conversion
int32_t framesToEcho = std::min(numInputFrames, numOutputFrames);
float *inputFloat = (float *)inputData;
float *outputFloat = (float *)outputData;
auto *inputFloat = const_cast<float *>(inputData);
float *outputFloat = outputData;
// zero out entire output array
memset(outputFloat, 0, numOutputFrames * getOutputStream()->getBytesPerFrame());
memset(outputFloat, 0, static_cast<size_t>(numOutputFrames)
* static_cast<size_t>(getOutputStream()->getBytesPerFrame()));
int32_t inputStride = getInputStream()->getChannelCount();
int32_t outputStride = getOutputStream()->getChannelCount();
float delayFrames = mDelayTimeSeconds * getOutputStream()->getSampleRate();
while (framesToEcho-- > 0) {
*outputFloat = mDelayLine->process(delayFrames, *inputFloat); // mono delay
for (int iChannel = 0; iChannel < inputStride; iChannel++) {
float sample = * (inputFloat + iChannel);
mPeakDetectors[iChannel].process(sample);
}
inputFloat += inputStride;
outputFloat += outputStride;
}
@@ -21,28 +21,31 @@
#include <sys/types.h>
#include "oboe/Oboe.h"
#include "FullDuplexStream.h"
#include "analyzer/LatencyAnalyzer.h"
#include "FullDuplexStreamWithConversion.h"
#include "InterpolatingDelayLine.h"
class FullDuplexEcho : public FullDuplexStream {
class FullDuplexEcho : public FullDuplexStreamWithConversion {
public:
FullDuplexEcho() {
setMNumInputBurstsCushion(0);
setNumInputBurstsCushion(0);
}
/**
* Called when data is available on both streams.
* Caller should override this method.
*/
oboe::DataCallbackResult onBothStreamsReady(
const void *inputData,
oboe::DataCallbackResult onBothStreamsReadyFloat(
const float *inputData,
int numInputFrames,
void *outputData,
float *outputData,
int numOutputFrames
) override;
oboe::Result start() override;
double getPeakLevel(int index);
void setDelayTime(double delayTimeSeconds) {
mDelayTimeSeconds = delayTimeSeconds;
}
@@ -51,6 +54,9 @@ private:
std::unique_ptr<InterpolatingDelayLine> mDelayLine;
static constexpr double kMaxDelayTimeSeconds = 4.0;
double mDelayTimeSeconds = kMaxDelayTimeSeconds;
std::atomic<int32_t> mNumChannels{0};
std::unique_ptr<PeakDetector[]> mPeakDetectors;
};
@@ -1,53 +0,0 @@
/*
* Copyright 2019 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#ifndef OBOETESTER_FULL_DUPLEX_GLITCHES_H
#define OBOETESTER_FULL_DUPLEX_GLITCHES_H
#include <unistd.h>
#include <sys/types.h>
#include "oboe/Oboe.h"
#include "FullDuplexAnalyzer.h"
#include "analyzer/GlitchAnalyzer.h"
class FullDuplexGlitches : public FullDuplexAnalyzer {
public:
FullDuplexGlitches() {
setMNumInputBurstsCushion(1);
}
bool isDone() {
return false;
}
GlitchAnalyzer *getGlitchAnalyzer() {
return &mGlitchAnalyzer;
}
LoopbackProcessor *getLoopbackProcessor() override {
return (LoopbackProcessor *) &mGlitchAnalyzer;
}
private:
GlitchAnalyzer mGlitchAnalyzer;
};
#endif //OBOETESTER_FULL_DUPLEX_GLITCHES_H
@@ -1,44 +0,0 @@
/*
* Copyright 2019 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#include <thread>
#include "common/OboeDebug.h"
#include "FullDuplexLatency.h"
static void analyze_data(FullDuplexLatency *fullDuplexLatency) {
fullDuplexLatency->analyzeData();
}
oboe::DataCallbackResult FullDuplexLatency::onBothStreamsReady(
const void *inputData,
int numInputFrames,
void *outputData,
int numOutputFrames) {
oboe::DataCallbackResult callbackResult = FullDuplexAnalyzer::onBothStreamsReady(
inputData, numInputFrames, outputData, numOutputFrames);
// Are we done?
if (mEchoAnalyzer.hasEnoughData()) {
// Crunch the numbers on a separate thread.
std::thread t(analyze_data, this);
t.detach();
callbackResult = oboe::DataCallbackResult::Stop;
}
return callbackResult;
};
@@ -1,65 +0,0 @@
/*
* Copyright 2019 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#ifndef OBOETESTER_FULL_DUPLEX_LATENCY_H
#define OBOETESTER_FULL_DUPLEX_LATENCY_H
#include <unistd.h>
#include <sys/types.h>
#include "oboe/Oboe.h"
#include "FullDuplexAnalyzer.h"
class FullDuplexLatency : public FullDuplexAnalyzer {
public:
FullDuplexLatency() {}
/**
* Called when data is available on both streams.
* Caller should override this method.
*/
oboe::DataCallbackResult onBothStreamsReady(
const void *inputData,
int numInputFrames,
void *outputData,
int numOutputFrames
) override;
bool isDone() {
return mEchoAnalyzer.isDone();
}
void analyzeData() {
mEchoAnalyzer.analyze();
}
LatencyAnalyzer *getLatencyAnalyzer() {
return &mEchoAnalyzer;
}
LoopbackProcessor *getLoopbackProcessor() override {
return (LoopbackProcessor *) &mEchoAnalyzer;
}
private:
PulseLatencyAnalyzer mEchoAnalyzer;
};
#endif //OBOETESTER_FULL_DUPLEX_LATENCY_H
@@ -1,137 +0,0 @@
/*
* Copyright 2019 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#include "common/OboeDebug.h"
#include "FullDuplexStream.h"
oboe::DataCallbackResult FullDuplexStream::onAudioReady(
oboe::AudioStream *outputStream,
void *audioData,
int numFrames) {
oboe::DataCallbackResult callbackResult = oboe::DataCallbackResult::Continue;
int32_t actualFramesRead = 0;
// Silence the output.
int32_t numBytes = numFrames * outputStream->getBytesPerFrame();
memset(audioData, 0 /* value */, numBytes);
if (mCountCallbacksToDrain > 0) {
// Drain the input.
int32_t totalFramesRead = 0;
do {
oboe::ResultWithValue<int32_t> result = getInputStream()->read(mInputBuffer.get(),
numFrames,
0 /* timeout */);
if (!result) {
// Ignore errors because input stream may not be started yet.
break;
}
actualFramesRead = result.value();
totalFramesRead += actualFramesRead;
} while (actualFramesRead > 0);
// Only counts if we actually got some data.
if (totalFramesRead > 0) {
mCountCallbacksToDrain--;
}
} else if (mCountInputBurstsCushion > 0) {
// Let the input fill up a bit so we are not so close to the write pointer.
mCountInputBurstsCushion--;
} else if (mCountCallbacksToDiscard > 0) {
mCountCallbacksToDiscard--;
// Ignore. Allow the input to reach to equilibrium with the output.
oboe::ResultWithValue<int32_t> resultAvailable = getInputStream()->getAvailableFrames();
if (!resultAvailable) {
LOGE("%s() getAvailableFrames() returned %s\n",
__func__, convertToText(resultAvailable.error()));
callbackResult = oboe::DataCallbackResult::Stop;
} else {
int32_t framesAvailable = resultAvailable.value();
if (framesAvailable >= mMinimumFramesBeforeRead) {
oboe::ResultWithValue<int32_t> resultRead = getInputStream()->read(mInputBuffer.get(), numFrames, 0 /* timeout */);
if (!resultRead) {
LOGE("%s() read() returned %s\n", __func__, convertToText(resultRead.error()));
callbackResult = oboe::DataCallbackResult::Stop;
}
}
}
} else {
int32_t framesRead = 0;
oboe::ResultWithValue<int32_t> resultAvailable = getInputStream()->getAvailableFrames();
if (!resultAvailable) {
LOGE("%s() getAvailableFrames() returned %s\n", __func__, convertToText(resultAvailable.error()));
callbackResult = oboe::DataCallbackResult::Stop;
} else {
int32_t framesAvailable = resultAvailable.value();
if (framesAvailable >= mMinimumFramesBeforeRead) {
// Read data into input buffer.
oboe::ResultWithValue<int32_t> resultRead = getInputStream()->read(mInputBuffer.get(), numFrames, 0 /* timeout */);
if (!resultRead) {
LOGE("%s() read() returned %s\n", __func__, convertToText(resultRead.error()));
callbackResult = oboe::DataCallbackResult::Stop;
} else {
framesRead = resultRead.value();
}
}
}
if (callbackResult == oboe::DataCallbackResult::Continue) {
callbackResult = onBothStreamsReady(
mInputBuffer.get(), framesRead,
audioData, numFrames);
}
}
if (callbackResult == oboe::DataCallbackResult::Stop) {
getInputStream()->requestStop();
}
return callbackResult;
}
oboe::Result FullDuplexStream::start() {
mCountCallbacksToDrain = kNumCallbacksToDrain;
mCountInputBurstsCushion = mNumInputBurstsCushion;
mCountCallbacksToDiscard = kNumCallbacksToDiscard;
// Determine maximum size that could possibly be called.
int32_t bufferSize = getOutputStream()->getBufferCapacityInFrames()
* getOutputStream()->getChannelCount();
if (bufferSize > mBufferSize) {
mInputBuffer = std::make_unique<float[]>(bufferSize);
mBufferSize = bufferSize;
}
oboe::Result result = getInputStream()->requestStart();
if (result != oboe::Result::OK) {
return result;
}
return getOutputStream()->requestStart();
}
oboe::Result FullDuplexStream::stop() {
getOutputStream()->requestStop(); // TODO result?
return getInputStream()->requestStop();
}
int32_t FullDuplexStream::getMNumInputBurstsCushion() const {
return mNumInputBurstsCushion;
}
void FullDuplexStream::setMNumInputBurstsCushion(int32_t numBursts) {
FullDuplexStream::mNumInputBurstsCushion = numBursts;
}
@@ -1,115 +0,0 @@
/*
* Copyright 2019 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#ifndef OBOETESTER_FULL_DUPLEX_STREAM_H
#define OBOETESTER_FULL_DUPLEX_STREAM_H
#include <unistd.h>
#include <sys/types.h>
#include "oboe/Oboe.h"
class FullDuplexStream : public oboe::AudioStreamCallback {
public:
FullDuplexStream() {}
virtual ~FullDuplexStream() = default;
void setInputStream(oboe::AudioStream *stream) {
mInputStream = stream;
}
oboe::AudioStream *getInputStream() {
return mInputStream;
}
void setOutputStream(oboe::AudioStream *stream) {
mOutputStream = stream;
}
oboe::AudioStream *getOutputStream() {
return mOutputStream;
}
virtual oboe::Result start();
virtual oboe::Result stop();
/**
* Called when data is available on both streams.
* Caller should override this method.
*/
virtual oboe::DataCallbackResult onBothStreamsReady(
const void *inputData,
int numInputFrames,
void *outputData,
int numOutputFrames
) = 0;
/**
* Called by Oboe when the stream is ready to process audio.
*/
oboe::DataCallbackResult onAudioReady(
oboe::AudioStream *audioStream,
void *audioData,
int numFrames) override;
int32_t getMNumInputBurstsCushion() const;
/**
* Number of bursts to leave in the input buffer as a cushion.
* Typically 0 for latency measurements
* or 1 for glitch tests.
*
* @param mNumInputBurstsCushion
*/
void setMNumInputBurstsCushion(int32_t mNumInputBurstsCushion);
void setMinimumFramesBeforeRead(int32_t numFrames) {
mMinimumFramesBeforeRead = numFrames;
}
int32_t getMinimumFramesBeforeRead() const {
return mMinimumFramesBeforeRead;
}
private:
// TODO add getters and setters
static constexpr int32_t kNumCallbacksToDrain = 20;
static constexpr int32_t kNumCallbacksToDiscard = 30;
// let input fill back up, usually 0 or 1
int32_t mNumInputBurstsCushion = 0;
int32_t mMinimumFramesBeforeRead = 0;
// We want to reach a state where the input buffer is empty and
// the output buffer is full.
// These are used in order.
// Drain several callback so that input is empty.
int32_t mCountCallbacksToDrain = kNumCallbacksToDrain;
// Let the input fill back up slightly so we don't run dry.
int32_t mCountInputBurstsCushion = mNumInputBurstsCushion;
// Discard some callbacks so the input and output reach equilibrium.
int32_t mCountCallbacksToDiscard = kNumCallbacksToDiscard;
oboe::AudioStream *mInputStream = nullptr;
oboe::AudioStream *mOutputStream = nullptr;
int32_t mBufferSize = 0;
std::unique_ptr<float[]> mInputBuffer;
};
#endif //OBOETESTER_FULL_DUPLEX_STREAM_H
@@ -0,0 +1,61 @@
/*
* Copyright 2023 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#include "common/OboeDebug.h"
#include "FullDuplexStreamWithConversion.h"
oboe::Result FullDuplexStreamWithConversion::start() {
// Determine maximum size that could possibly be called.
int32_t maxFrames = getOutputStream()->getBufferCapacityInFrames();
int32_t inputBufferSize = maxFrames * getInputStream()->getChannelCount();
int32_t outputBufferSize = maxFrames * getOutputStream()->getChannelCount();
mInputConverter = std::make_unique<FormatConverterBox>(inputBufferSize,
getInputStream()->getFormat(),
oboe::AudioFormat::Float);
mOutputConverter = std::make_unique<FormatConverterBox>(outputBufferSize,
oboe::AudioFormat::Float,
getOutputStream()->getFormat());
return FullDuplexStream::start();
}
oboe::ResultWithValue<int32_t> FullDuplexStreamWithConversion::readInput(int32_t numFrames) {
oboe::ResultWithValue<int32_t> result = getInputStream()->read(
mInputConverter->getInputBuffer(),
numFrames,
0 /* timeout */);
if (result == oboe::Result::OK) {
int32_t numSamples = result.value() * getInputStream()->getChannelCount();
mInputConverter->convertInternalBuffers(numSamples);
}
return result;
}
oboe::DataCallbackResult FullDuplexStreamWithConversion::onBothStreamsReady(
const void *inputData,
int numInputFrames,
void *outputData,
int numOutputFrames
) {
oboe::DataCallbackResult callbackResult = oboe::DataCallbackResult::Continue;
callbackResult = onBothStreamsReadyFloat(
static_cast<const float *>(mInputConverter->getOutputBuffer()),
numInputFrames,
static_cast<float *>(mOutputConverter->getInputBuffer()),
numOutputFrames);
mOutputConverter->convertFromInternalInput( outputData,
numOutputFrames * getOutputStream()->getChannelCount());
return callbackResult;
}
@@ -0,0 +1,61 @@
/*
* Copyright 2023 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#ifndef OBOETESTER_FULL_DUPLEX_STREAM_WITH_CONVERSION_H
#define OBOETESTER_FULL_DUPLEX_STREAM_WITH_CONVERSION_H
#include <unistd.h>
#include <sys/types.h>
#include "oboe/Oboe.h"
#include "FormatConverterBox.h"
class FullDuplexStreamWithConversion : public oboe::FullDuplexStream {
public:
/**
* Called when data is available on both streams.
* Caller must override this method.
*/
virtual oboe::DataCallbackResult onBothStreamsReadyFloat(
const float *inputData,
int numInputFrames,
float *outputData,
int numOutputFrames
) = 0;
/**
* Overrides the default onBothStreamsReady by converting to floats and then calling
* onBothStreamsReadyFloat().
*/
oboe::DataCallbackResult onBothStreamsReady(
const void *inputData,
int numInputFrames,
void *outputData,
int numOutputFrames
) override;
oboe::ResultWithValue<int32_t> readInput(int32_t numFrames) override;
virtual oboe::Result start() override;
private:
std::unique_ptr<FormatConverterBox> mInputConverter;
std::unique_ptr<FormatConverterBox> mOutputConverter;
};
#endif //OBOETESTER_FULL_DUPLEX_STREAM_WITH_CONVERSION_H
@@ -17,37 +17,35 @@
#include "common/OboeDebug.h"
#include "InputStreamCallbackAnalyzer.h"
double InputStreamCallbackAnalyzer::getPeakLevel(int index) {
if (mPeakDetectors == nullptr) {
LOGE("%s() called before setup()", __func__);
return -1.0;
} else if (index < 0 || index >= mNumChannels) {
LOGE("%s(), index out of range, 0 <= %d < %d", __func__, index, mNumChannels);
return -2.0;
}
return mPeakDetectors[index].getLevel();
}
oboe::DataCallbackResult InputStreamCallbackAnalyzer::onAudioReady(
oboe::AudioStream *audioStream,
void *audioData,
int numFrames) {
int32_t channelCount = audioStream->getChannelCount();
if (audioStream->getFormat() == oboe::AudioFormat::I16) {
int16_t *shortData = (int16_t *) audioData;
if (mRecording != nullptr) {
mRecording->write(shortData, numFrames);
}
int16_t *frameData = shortData;
for (int iFrame = 0; iFrame < numFrames; iFrame++) {
for (int iChannel = 0; iChannel < channelCount; iChannel++) {
float sample = frameData[iChannel] / 32768.0f;
mPeakDetectors[iChannel].process(sample);
}
frameData += channelCount;
}
} else if (audioStream->getFormat() == oboe::AudioFormat::Float) {
float *floatData = (float *) audioData;
if (mRecording != nullptr) {
mRecording->write(floatData, numFrames);
}
float *frameData = floatData;
for (int iFrame = 0; iFrame < numFrames; iFrame++) {
for (int iChannel = 0; iChannel < channelCount; iChannel++) {
float sample = frameData[iChannel];
mPeakDetectors[iChannel].process(sample);
}
frameData += channelCount;
maybeHang(getNanoseconds());
printScheduler();
mInputConverter->convertToInternalOutput(numFrames * channelCount, audioData);
float *floatData = (float *) mInputConverter->getOutputBuffer();
if (mRecording != nullptr) {
mRecording->write(floatData, numFrames);
}
int32_t sampleIndex = 0;
for (int iFrame = 0; iFrame < numFrames; iFrame++) {
for (int iChannel = 0; iChannel < channelCount; iChannel++) {
float sample = floatData[sampleIndex++];
mPeakDetectors[iChannel].process(sample);
}
}
@@ -23,18 +23,31 @@
// TODO #include "flowgraph/FlowGraph.h"
#include "oboe/Oboe.h"
#include "MultiChannelRecording.h"
#include "analyzer/PeakDetector.h"
#include "FormatConverterBox.h"
#include "MultiChannelRecording.h"
#include "OboeTesterStreamCallback.h"
constexpr int kMaxInputChannels = 8;
class InputStreamCallbackAnalyzer : public oboe::AudioStreamCallback {
class InputStreamCallbackAnalyzer : public OboeTesterStreamCallback {
public:
void reset() {
for (auto detector : mPeakDetectors) {
detector.reset();
for (int iChannel = 0; iChannel < mNumChannels; iChannel++) {
mPeakDetectors[iChannel].reset();
}
OboeTesterStreamCallback::reset();
}
void setup(int32_t maxFramesPerCallback,
int32_t channelCount,
oboe::AudioFormat inputFormat) {
mNumChannels = channelCount;
mPeakDetectors = std::make_unique<PeakDetector[]>(channelCount);
int32_t bufferSize = maxFramesPerCallback * channelCount;
mInputConverter = std::make_unique<FormatConverterBox>(bufferSize,
inputFormat,
oboe::AudioFormat::Float);
}
/**
@@ -49,9 +62,7 @@ public:
mRecording = recording;
}
double getPeakLevel(int index) {
return mPeakDetectors[index].getLevel();
}
double getPeakLevel(int index);
void setMinimumFramesBeforeRead(int32_t numFrames) {
mMinimumFramesBeforeRead = numFrames;
@@ -62,11 +73,13 @@ public:
}
public:
PeakDetector mPeakDetectors[kMaxInputChannels];
MultiChannelRecording *mRecording = nullptr;
int32_t mNumChannels = 0;
std::unique_ptr<PeakDetector[]> mPeakDetectors;
MultiChannelRecording *mRecording = nullptr;
private:
int32_t mMinimumFramesBeforeRead = 0;
std::unique_ptr<FormatConverterBox> mInputConverter;
int32_t mMinimumFramesBeforeRead = 0;
};
#endif //NATIVEOBOE_INPUTSTREAMCALLBACKANALYZER_H
@@ -14,7 +14,8 @@
* limitations under the License.
*/
#include "common/OboeDebug.h"
#include <algorithm>
#include "InterpolatingDelayLine.h"
InterpolatingDelayLine::InterpolatingDelayLine(int32_t delaySize) {
@@ -21,9 +21,6 @@
#include <unistd.h>
#include <sys/types.h>
#include "oboe/Oboe.h"
#include "FullDuplexStream.h"
/**
* Monophonic delay line.
*/
@@ -14,10 +14,20 @@
* limitations under the License.
*/
// Set to 1 for debugging race condition #1180 with mAAudioStream.
// See also AudioStreamAAudio.cpp in Oboe.
// This was left in the code so that we could test the fix again easily in the future.
// We could not trigger the race condition without adding these get calls and the sleeps.
#define DEBUG_CLOSE_RACE 0
#include <fstream>
#include <iostream>
#if DEBUG_CLOSE_RACE
#include <thread>
#endif // DEBUG_CLOSE_RACE
#include <vector>
#include "oboe/AudioClock.h"
#include "util/WaveFileWriter.h"
#include "NativeAudioContext.h"
@@ -57,9 +67,9 @@ private:
bool ActivityContext::mUseCallback = true;
int ActivityContext::callbackSize = 0;
oboe::AudioStream * ActivityContext::getOutputStream() {
std::shared_ptr<oboe::AudioStream> ActivityContext::getOutputStream() {
for (auto entry : mOboeStreams) {
oboe::AudioStream *oboeStream = entry.second.get();
std::shared_ptr<oboe::AudioStream> oboeStream = entry.second;
if (oboeStream->getDirection() == oboe::Direction::Output) {
return oboeStream;
}
@@ -67,9 +77,9 @@ oboe::AudioStream * ActivityContext::getOutputStream() {
return nullptr;
}
oboe::AudioStream * ActivityContext::getInputStream() {
std::shared_ptr<oboe::AudioStream> ActivityContext::getInputStream() {
for (auto entry : mOboeStreams) {
oboe::AudioStream *oboeStream = entry.second.get();
std::shared_ptr<oboe::AudioStream> oboeStream = entry.second;
if (oboeStream != nullptr) {
if (oboeStream->getDirection() == oboe::Direction::Input) {
return oboeStream;
@@ -88,9 +98,19 @@ int32_t ActivityContext::allocateStreamIndex() {
return mNextStreamHandle++;
}
oboe::Result ActivityContext::release() {
oboe::Result result = oboe::Result::OK;
stopBlockingIOThread();
for (auto entry : mOboeStreams) {
std::shared_ptr<oboe::AudioStream> oboeStream = entry.second;
result = oboeStream->release();
}
return result;
}
void ActivityContext::close(int32_t streamIndex) {
stopBlockingIOThread();
oboe::AudioStream *oboeStream = getStream(streamIndex);
std::shared_ptr<oboe::AudioStream> oboeStream = getStream(streamIndex);
if (oboeStream != nullptr) {
oboeStream->close();
LOGD("ActivityContext::%s() delete stream %d ", __func__, streamIndex);
@@ -99,19 +119,18 @@ void ActivityContext::close(int32_t streamIndex) {
}
bool ActivityContext::isMMapUsed(int32_t streamIndex) {
oboe::AudioStream *oboeStream = getStream(streamIndex);
std::shared_ptr<oboe::AudioStream> oboeStream = getStream(streamIndex);
if (oboeStream == nullptr) return false;
if (oboeStream->getAudioApi() != AudioApi::AAudio) return false;
return AAudioExtensions::getInstance().isMMapUsed(oboeStream);
return AAudioExtensions::getInstance().isMMapUsed(oboeStream.get());
}
oboe::Result ActivityContext::pause() {
oboe::Result result = oboe::Result::OK;
stopBlockingIOThread();
for (auto entry : mOboeStreams) {
oboe::AudioStream *oboeStream = entry.second.get();
std::shared_ptr<oboe::AudioStream> oboeStream = entry.second;
result = oboeStream->requestPause();
printScheduler();
}
return result;
}
@@ -120,9 +139,8 @@ oboe::Result ActivityContext::stopAllStreams() {
oboe::Result result = oboe::Result::OK;
stopBlockingIOThread();
for (auto entry : mOboeStreams) {
oboe::AudioStream *oboeStream = entry.second.get();
std::shared_ptr<oboe::AudioStream> oboeStream = entry.second;
result = oboeStream->requestStop();
printScheduler();
}
return result;
}
@@ -130,23 +148,23 @@ oboe::Result ActivityContext::stopAllStreams() {
void ActivityContext::configureBuilder(bool isInput, oboe::AudioStreamBuilder &builder) {
// We needed the proxy because we did not know the channelCount when we setup the Builder.
if (mUseCallback) {
LOGD("ActivityContext::open() set callback to use oboeCallbackProxy, callback size = %d",
callbackSize);
builder.setCallback(&oboeCallbackProxy);
builder.setFramesPerCallback(callbackSize);
builder.setDataCallback(&oboeCallbackProxy);
}
}
int ActivityContext::open(jint nativeApi,
jint sampleRate,
jint channelCount,
jint channelMask,
jint format,
jint sharingMode,
jint performanceMode,
jint inputPreset,
jint usage,
jint contentType,
jint bufferCapacityInFrames,
jint deviceId,
jint sessionId,
jint framesPerBurst,
jboolean channelConversionAllowed,
jboolean formatConversionAllowed,
jint rateConversionQuality,
@@ -182,6 +200,9 @@ int ActivityContext::open(jint nativeApi,
->setSharingMode((oboe::SharingMode) sharingMode)
->setPerformanceMode((oboe::PerformanceMode) performanceMode)
->setInputPreset((oboe::InputPreset)inputPreset)
->setUsage((oboe::Usage)usage)
->setContentType((oboe::ContentType)contentType)
->setBufferCapacityInFrames(bufferCapacityInFrames)
->setDeviceId(deviceId)
->setSessionId((oboe::SessionId) sessionId)
->setSampleRate(sampleRate)
@@ -190,7 +211,13 @@ int ActivityContext::open(jint nativeApi,
->setFormatConversionAllowed(formatConversionAllowed)
->setSampleRateConversionQuality((oboe::SampleRateConversionQuality) rateConversionQuality)
;
if (channelMask != (jint) oboe::ChannelMask::Unspecified) {
// Set channel mask when it is specified.
builder.setChannelMask((oboe::ChannelMask) channelMask);
}
if (mUseCallback) {
builder.setFramesPerCallback(callbackSize);
}
configureBuilder(isInput, builder);
builder.setAudioApi(audioApi);
@@ -199,6 +226,12 @@ int ActivityContext::open(jint nativeApi,
bool oldMMapEnabled = AAudioExtensions::getInstance().isMMapEnabled();
AAudioExtensions::getInstance().setMMapEnabled(isMMap);
// Record time for opening.
if (isInput) {
mInputOpenedAt = oboe::AudioClock::getNanoseconds();
} else {
mOutputOpenedAt = oboe::AudioClock::getNanoseconds();
}
// Open a stream based on the builder settings.
std::shared_ptr<oboe::AudioStream> oboeStream;
Result result = builder.openStream(oboeStream);
@@ -215,7 +248,7 @@ int ActivityContext::open(jint nativeApi,
createRecording();
finishOpen(isInput, oboeStream.get());
finishOpen(isInput, oboeStream);
}
if (!mUseCallback) {
@@ -223,20 +256,24 @@ int ActivityContext::open(jint nativeApi,
dataBuffer = std::make_unique<float[]>(numSamples);
}
return (result != Result::OK) ? (int)result : streamIndex;
if (result != Result::OK) {
return (int) result;
} else {
configureAfterOpen();
return streamIndex;
}
}
oboe::Result ActivityContext::start() {
oboe::Result result = oboe::Result::OK;
oboe::AudioStream *inputStream = getInputStream();
oboe::AudioStream *outputStream = getOutputStream();
std::shared_ptr<oboe::AudioStream> inputStream = getInputStream();
std::shared_ptr<oboe::AudioStream> outputStream = getOutputStream();
if (inputStream == nullptr && outputStream == nullptr) {
LOGD("%s() - no streams defined", __func__);
return oboe::Result::ErrorInvalidState; // not open
}
configureForStart();
audioStreamGateway.reset();
result = startStreams();
if (!mUseCallback && result == oboe::Result::OK) {
@@ -245,6 +282,29 @@ oboe::Result ActivityContext::start() {
dataThread = new std::thread(threadCallback, this);
}
#if DEBUG_CLOSE_RACE
// Also put a sleep for 400 msec in AudioStreamAAudio::updateFramesRead().
if (outputStream != nullptr) {
std::thread raceDebugger([outputStream]() {
while (outputStream->getState() != StreamState::Closed) {
int64_t framesRead = outputStream->getFramesRead();
LOGD("raceDebugger, framesRead = %d, state = %d",
(int) framesRead, (int) outputStream->getState());
}
});
raceDebugger.detach();
}
#endif // DEBUG_CLOSE_RACE
return result;
}
oboe::Result ActivityContext::flush() {
oboe::Result result = oboe::Result::OK;
for (auto entry : mOboeStreams) {
std::shared_ptr<oboe::AudioStream> oboeStream = entry.second;
result = oboeStream->requestFlush();
}
return result;
}
@@ -259,10 +319,11 @@ int32_t ActivityContext::saveWaveFile(const char *filename) {
}
MyOboeOutputStream outStream;
WaveFileWriter writer(&outStream);
// You must setup the format before the first write().
writer.setFrameRate(mSampleRate);
writer.setSamplesPerFrame(mRecording->getChannelCount());
writer.setBitsPerSample(24);
writer.setFrameCount(mRecording->getSizeInFrames());
float buffer[mRecording->getChannelCount()];
// Read samples from start to finish.
mRecording->rewind();
@@ -283,13 +344,25 @@ int32_t ActivityContext::saveWaveFile(const char *filename) {
return outStream.length();
}
double ActivityContext::getTimestampLatency(int32_t streamIndex) {
std::shared_ptr<oboe::AudioStream> oboeStream = getStream(streamIndex);
if (oboeStream != nullptr) {
auto result = oboeStream->calculateLatencyMillis();
return (!result) ? -1.0 : result.value();
}
return -1.0;
}
// =================================================================== ActivityTestOutput
void ActivityTestOutput::close(int32_t streamIndex) {
ActivityContext::close(streamIndex);
manyToMulti.reset(nullptr);
monoToMulti.reset(nullptr);
mVolumeRamp.reset();
mSinkFloat.reset();
mSinkI16.reset();
mSinkI24.reset();
mSinkI32.reset();
}
void ActivityTestOutput::setChannelEnabled(int channelIndex, bool enabled) {
@@ -313,6 +386,9 @@ void ActivityTestOutput::setChannelEnabled(int channelIndex, bool enabled) {
mExponentialShape.output.connect(&sineOscillators[channelIndex].frequency);
sineOscillators[channelIndex].output.connect(manyToMulti->inputs[channelIndex].get());
break;
case SignalType::WhiteNoise:
mWhiteNoise.output.connect(manyToMulti->inputs[channelIndex].get());
break;
default:
break;
}
@@ -321,13 +397,20 @@ void ActivityTestOutput::setChannelEnabled(int channelIndex, bool enabled) {
}
}
void ActivityTestOutput::configureForStart() {
void ActivityTestOutput::configureAfterOpen() {
manyToMulti = std::make_unique<ManyToMultiConverter>(mChannelCount);
mSinkFloat = std::make_unique<SinkFloat>(mChannelCount);
mSinkI16 = std::make_unique<SinkI16>(mChannelCount);
std::shared_ptr<oboe::AudioStream> outputStream = getOutputStream();
oboe::AudioStream *outputStream = getOutputStream();
mVolumeRamp = std::make_shared<RampLinear>(mChannelCount);
mVolumeRamp->setLengthInFrames(kRampMSec * outputStream->getSampleRate() /
MILLISECONDS_PER_SECOND);
mVolumeRamp->setTarget(mAmplitude);
mSinkFloat = std::make_shared<SinkFloat>(mChannelCount);
mSinkI16 = std::make_shared<SinkI16>(mChannelCount);
mSinkI24 = std::make_shared<SinkI24>(mChannelCount);
mSinkI32 = std::make_shared<SinkI32>(mChannelCount);
mTriangleOscillator.setSampleRate(outputStream->getSampleRate());
mTriangleOscillator.frequency.setValue(1.0/kSweepPeriod);
@@ -344,35 +427,52 @@ void ActivityTestOutput::configureForStart() {
mTriangleOscillator.output.connect(&(mExponentialShape.input));
{
double frequency = 330.0;
// Go up by a minor third or a perfect fourth just intoned interval.
const float interval = (mChannelCount > 8) ? (6.0f / 5.0f) : (4.0f / 3.0f);
for (int i = 0; i < mChannelCount; i++) {
sineOscillators[i].setSampleRate(outputStream->getSampleRate());
sineOscillators[i].frequency.setValue(frequency);
frequency *= 4.0 / 3.0; // each sine is at a higher frequency
sineOscillators[i].amplitude.setValue(AMPLITUDE_SINE);
sawtoothOscillators[i].setSampleRate(outputStream->getSampleRate());
sawtoothOscillators[i].frequency.setValue(frequency);
sawtoothOscillators[i].amplitude.setValue(AMPLITUDE_SAWTOOTH);
frequency *= interval; // each wave is at a higher frequency
setChannelEnabled(i, true);
}
}
manyToMulti->output.connect(&(mSinkFloat.get()->input));
manyToMulti->output.connect(&(mSinkI16.get()->input));
mWhiteNoise.amplitude.setValue(0.5);
manyToMulti->output.connect(&(mVolumeRamp.get()->input));
mVolumeRamp->output.connect(&(mSinkFloat.get()->input));
mVolumeRamp->output.connect(&(mSinkI16.get()->input));
mVolumeRamp->output.connect(&(mSinkI24.get()->input));
mVolumeRamp->output.connect(&(mSinkI32.get()->input));
// Clear framePosition in sine oscillators.
mSinkFloat->pullReset();
mSinkI16->pullReset();
mSinkI24->pullReset();
mSinkI32->pullReset();
configureStreamGateway();
}
void ActivityTestOutput::configureStreamGateway() {
oboe::AudioStream *outputStream = getOutputStream();
std::shared_ptr<oboe::AudioStream> outputStream = getOutputStream();
if (outputStream->getFormat() == oboe::AudioFormat::I16) {
audioStreamGateway.setAudioSink(mSinkI16);
} else if (outputStream->getFormat() == oboe::AudioFormat::I24) {
audioStreamGateway.setAudioSink(mSinkI24);
} else if (outputStream->getFormat() == oboe::AudioFormat::I32) {
audioStreamGateway.setAudioSink(mSinkI32);
} else if (outputStream->getFormat() == oboe::AudioFormat::Float) {
audioStreamGateway.setAudioSink(mSinkFloat);
}
if (mUseCallback) {
oboeCallbackProxy.setCallback(&audioStreamGateway);
oboeCallbackProxy.setDataCallback(&audioStreamGateway);
}
}
@@ -380,7 +480,7 @@ void ActivityTestOutput::runBlockingIO() {
int32_t framesPerBlock = getFramesPerBlock();
oboe::DataCallbackResult callbackResult = oboe::DataCallbackResult::Continue;
oboe::AudioStream *oboeStream = getOutputStream();
std::shared_ptr<oboe::AudioStream> oboeStream = getOutputStream();
if (oboeStream == nullptr) {
LOGE("%s() : no stream found\n", __func__);
return;
@@ -389,7 +489,7 @@ void ActivityTestOutput::runBlockingIO() {
while (threadEnabled.load()
&& callbackResult == oboe::DataCallbackResult::Continue) {
// generate output by calling the callback
callbackResult = audioStreamGateway.onAudioReady(oboeStream,
callbackResult = audioStreamGateway.onAudioReady(oboeStream.get(),
dataBuffer.get(),
framesPerBlock);
@@ -409,11 +509,22 @@ void ActivityTestOutput::runBlockingIO() {
}
}
oboe::Result ActivityTestOutput::startStreams() {
mSinkFloat->pullReset();
mSinkI16->pullReset();
mSinkI24->pullReset();
mSinkI32->pullReset();
if (mVolumeRamp != nullptr) {
mVolumeRamp->setTarget(mAmplitude);
}
return getOutputStream()->start();
}
// ======================================================================= ActivityTestInput
void ActivityTestInput::configureForStart() {
void ActivityTestInput::configureAfterOpen() {
mInputAnalyzer.reset();
if (mUseCallback) {
oboeCallbackProxy.setCallback(&mInputAnalyzer);
oboeCallbackProxy.setDataCallback(&mInputAnalyzer);
}
mInputAnalyzer.setRecording(mRecording.get());
}
@@ -422,7 +533,7 @@ void ActivityTestInput::runBlockingIO() {
int32_t framesPerBlock = getFramesPerBlock();
oboe::DataCallbackResult callbackResult = oboe::DataCallbackResult::Continue;
oboe::AudioStream *oboeStream = getInputStream();
std::shared_ptr<oboe::AudioStream> oboeStream = getInputStream();
if (oboeStream == nullptr) {
LOGE("%s() : no stream found\n", __func__);
return;
@@ -450,7 +561,7 @@ void ActivityTestInput::runBlockingIO() {
}
// analyze input
callbackResult = mInputAnalyzer.onAudioReady(oboeStream,
callbackResult = mInputAnalyzer.onAudioReady(oboeStream.get(),
dataBuffer.get(),
framesRead);
}
@@ -474,8 +585,7 @@ oboe::Result ActivityRecording::startPlayback() {
builder.setChannelCount(mChannelCount)
->setSampleRate(mSampleRate)
->setFormat(oboe::AudioFormat::Float)
->setCallback(&mPlayRecordingCallback)
->setAudioApi(oboe::AudioApi::OpenSLES);
->setCallback(&mPlayRecordingCallback);
oboe::Result result = builder.openStream(&playbackStream);
if (result != oboe::Result::OK) {
delete playbackStream;
@@ -491,13 +601,15 @@ oboe::Result ActivityRecording::startPlayback() {
}
// ======================================================================= ActivityTapToTone
void ActivityTapToTone::configureForStart() {
void ActivityTapToTone::configureAfterOpen() {
monoToMulti = std::make_unique<MonoToMultiConverter>(mChannelCount);
mSinkFloat = std::make_unique<SinkFloat>(mChannelCount);
mSinkI16 = std::make_unique<SinkI16>(mChannelCount);
mSinkFloat = std::make_shared<SinkFloat>(mChannelCount);
mSinkI16 = std::make_shared<SinkI16>(mChannelCount);
mSinkI24 = std::make_shared<SinkI24>(mChannelCount);
mSinkI32 = std::make_shared<SinkI32>(mChannelCount);
oboe::AudioStream *outputStream = getOutputStream();
std::shared_ptr<oboe::AudioStream> outputStream = getOutputStream();
sawPingGenerator.setSampleRate(outputStream->getSampleRate());
sawPingGenerator.frequency.setValue(FREQUENCY_SAW_PING);
sawPingGenerator.amplitude.setValue(AMPLITUDE_SAW_PING);
@@ -505,20 +617,26 @@ void ActivityTapToTone::configureForStart() {
sawPingGenerator.output.connect(&(monoToMulti->input));
monoToMulti->output.connect(&(mSinkFloat.get()->input));
monoToMulti->output.connect(&(mSinkI16.get()->input));
monoToMulti->output.connect(&(mSinkI24.get()->input));
monoToMulti->output.connect(&(mSinkI32.get()->input));
mSinkFloat->pullReset();
mSinkI16->pullReset();
mSinkI24->pullReset();
mSinkI32->pullReset();
sawPingGenerator.setEnabled(false);
configureStreamGateway();
}
// ======================================================================= ActivityRoundTripLatency
// ======================================================================= ActivityFullDuplex
void ActivityFullDuplex::configureBuilder(bool isInput, oboe::AudioStreamBuilder &builder) {
if (isInput) {
// Ideally the output streams should be opened first.
oboe::AudioStream *outputStream = getOutputStream();
std::shared_ptr<oboe::AudioStream> outputStream = getOutputStream();
if (outputStream != nullptr) {
// Make sure the capacity is bigger than two bursts.
int32_t burst = outputStream->getFramesPerBurst();
builder.setBufferCapacityInFrames(2 * burst);
// The input and output buffers will run in sync with input empty
// and output full. So set the input capacity to match the output.
builder.setBufferCapacityInFrames(outputStream->getBufferCapacityInFrames());
}
}
}
@@ -532,15 +650,16 @@ void ActivityEcho::configureBuilder(bool isInput, oboe::AudioStreamBuilder &buil
}
// only output uses a callback, input is polled
if (!isInput) {
builder.setCallback(mFullDuplexEcho.get());
builder.setCallback((oboe::AudioStreamCallback *) &oboeCallbackProxy);
oboeCallbackProxy.setDataCallback(mFullDuplexEcho.get());
}
}
void ActivityEcho::finishOpen(bool isInput, oboe::AudioStream *oboeStream) {
void ActivityEcho::finishOpen(bool isInput, std::shared_ptr<oboe::AudioStream> &oboeStream) {
if (isInput) {
mFullDuplexEcho->setInputStream(oboeStream);
mFullDuplexEcho->setSharedInputStream(oboeStream);
} else {
mFullDuplexEcho->setOutputStream(oboeStream);
mFullDuplexEcho->setSharedOutputStream(oboeStream);
}
}
@@ -549,45 +668,102 @@ void ActivityRoundTripLatency::configureBuilder(bool isInput, oboe::AudioStreamB
ActivityFullDuplex::configureBuilder(isInput, builder);
if (mFullDuplexLatency.get() == nullptr) {
mFullDuplexLatency = std::make_unique<FullDuplexLatency>();
mFullDuplexLatency = std::make_unique<FullDuplexAnalyzer>(mLatencyAnalyzer.get());
}
if (!isInput) {
// only output uses a callback, input is polled
builder.setCallback(mFullDuplexLatency.get());
builder.setCallback((oboe::AudioStreamCallback *) &oboeCallbackProxy);
oboeCallbackProxy.setDataCallback(mFullDuplexLatency.get());
}
}
void ActivityRoundTripLatency::finishOpen(bool isInput, oboe::AudioStream *oboeStream) {
void ActivityRoundTripLatency::finishOpen(bool isInput, std::shared_ptr<oboe::AudioStream>
&oboeStream) {
if (isInput) {
mFullDuplexLatency->setInputStream(oboeStream);
mFullDuplexLatency->setSharedInputStream(oboeStream);
mFullDuplexLatency->setRecording(mRecording.get());
} else {
mFullDuplexLatency->setOutputStream(oboeStream);
mFullDuplexLatency->setSharedOutputStream(oboeStream);
}
}
// The timestamp latency is the difference between the input
// and output times for a specific frame.
// Start with the position and time from an input timestamp.
// Map the input position to the corresponding position in output
// and calculate its time.
// Use the difference between framesWritten and framesRead to
// convert input positions to output positions.
jdouble ActivityRoundTripLatency::measureTimestampLatency() {
if (!mFullDuplexLatency->isWriteReadDeltaValid()) return -1.0;
int64_t writeReadDelta = mFullDuplexLatency->getWriteReadDelta();
auto inputTimestampResult = mFullDuplexLatency->getInputStream()->getTimestamp(CLOCK_MONOTONIC);
if (!inputTimestampResult) return -1.0;
auto outputTimestampResult = mFullDuplexLatency->getOutputStream()->getTimestamp(CLOCK_MONOTONIC);
if (!outputTimestampResult) return -1.0;
int64_t inputPosition = inputTimestampResult.value().position;
int64_t inputTimeNanos = inputTimestampResult.value().timestamp;
int64_t ouputPosition = outputTimestampResult.value().position;
int64_t outputTimeNanos = outputTimestampResult.value().timestamp;
// Map input frame position to the corresponding output frame.
int64_t mappedPosition = inputPosition + writeReadDelta;
// Calculate when that frame will play.
int32_t sampleRate = mFullDuplexLatency->getOutputStream()->getSampleRate();
int64_t mappedTimeNanos = outputTimeNanos + ((mappedPosition - ouputPosition) * 1e9) / sampleRate;
// Latency is the difference in time between when a frame was recorded and
// when its corresponding echo was played.
return (mappedTimeNanos - inputTimeNanos) * 1.0e-6; // convert nanos to millis
}
// ======================================================================= ActivityGlitches
void ActivityGlitches::configureBuilder(bool isInput, oboe::AudioStreamBuilder &builder) {
ActivityFullDuplex::configureBuilder(isInput, builder);
if (mFullDuplexGlitches.get() == nullptr) {
mFullDuplexGlitches = std::make_unique<FullDuplexGlitches>();
mFullDuplexGlitches = std::make_unique<FullDuplexAnalyzer>(&mGlitchAnalyzer);
}
if (!isInput) {
// only output uses a callback, input is polled
builder.setCallback(mFullDuplexGlitches.get());
builder.setCallback((oboe::AudioStreamCallback *) &oboeCallbackProxy);
oboeCallbackProxy.setDataCallback(mFullDuplexGlitches.get());
}
}
void ActivityGlitches::finishOpen(bool isInput, oboe::AudioStream *oboeStream) {
void ActivityGlitches::finishOpen(bool isInput, std::shared_ptr<oboe::AudioStream> &oboeStream) {
if (isInput) {
mFullDuplexGlitches->setInputStream(oboeStream);
mFullDuplexGlitches->setSharedInputStream(oboeStream);
mFullDuplexGlitches->setRecording(mRecording.get());
} else {
mFullDuplexGlitches->setOutputStream(oboeStream);
mFullDuplexGlitches->setSharedOutputStream(oboeStream);
}
}
// ======================================================================= ActivityDataPath
void ActivityDataPath::configureBuilder(bool isInput, oboe::AudioStreamBuilder &builder) {
ActivityFullDuplex::configureBuilder(isInput, builder);
if (mFullDuplexDataPath.get() == nullptr) {
mFullDuplexDataPath = std::make_unique<FullDuplexAnalyzer>(&mDataPathAnalyzer);
}
if (!isInput) {
// only output uses a callback, input is polled
builder.setCallback((oboe::AudioStreamCallback *) &oboeCallbackProxy);
oboeCallbackProxy.setDataCallback(mFullDuplexDataPath.get());
}
}
void ActivityDataPath::finishOpen(bool isInput, std::shared_ptr<oboe::AudioStream> &oboeStream) {
if (isInput) {
mFullDuplexDataPath->setSharedInputStream(oboeStream);
mFullDuplexDataPath->setRecording(mRecording.get());
} else {
mFullDuplexDataPath->setSharedOutputStream(oboeStream);
}
}
// =================================================================== ActivityTestDisconnect
void ActivityTestDisconnect::close(int32_t streamIndex) {
@@ -595,9 +771,9 @@ void ActivityTestDisconnect::close(int32_t streamIndex) {
mSinkFloat.reset();
}
void ActivityTestDisconnect::configureForStart() {
oboe::AudioStream *outputStream = getOutputStream();
oboe::AudioStream *inputStream = getInputStream();
void ActivityTestDisconnect::configureAfterOpen() {
std::shared_ptr<oboe::AudioStream> outputStream = getOutputStream();
std::shared_ptr<oboe::AudioStream> inputStream = getInputStream();
if (outputStream) {
mSinkFloat = std::make_unique<SinkFloat>(mChannelCount);
sineOscillator = std::make_unique<SineOscillator>();
@@ -607,13 +783,13 @@ void ActivityTestDisconnect::configureForStart() {
sineOscillator->frequency.setValue(440.0);
sineOscillator->amplitude.setValue(AMPLITUDE_SINE);
sineOscillator->output.connect(&(monoToMulti->input));
monoToMulti->output.connect(&(mSinkFloat->input));
// Clear framePosition in sine oscillators.
mSinkFloat->pullReset();
audioStreamGateway.setAudioSink(mSinkFloat);
} else if (inputStream) {
audioStreamGateway.setAudioSink(nullptr);
}
oboeCallbackProxy.setCallback(&audioStreamGateway);
oboeCallbackProxy.setDataCallback(&audioStreamGateway);
}
@@ -17,7 +17,6 @@
#ifndef NATIVEOBOE_NATIVEAUDIOCONTEXT_H
#define NATIVEOBOE_NATIVEAUDIOCONTEXT_H
#include <dlfcn.h>
#include <jni.h>
#include <sys/system_properties.h>
#include <thread>
@@ -27,161 +26,50 @@
#include "common/OboeDebug.h"
#include "oboe/Oboe.h"
#include "aaudio/AAudioExtensions.h"
#include "AudioStreamGateway.h"
#include "flowunits/ImpulseOscillator.h"
#include "flowgraph/ManyToMultiConverter.h"
#include "flowgraph/MonoToMultiConverter.h"
#include "flowgraph/RampLinear.h"
#include "flowgraph/SinkFloat.h"
#include "flowgraph/SinkI16.h"
#include "flowgraph/SinkI24.h"
#include "flowgraph/SinkI32.h"
#include "flowunits/ExponentialShape.h"
#include "flowunits/LinearShape.h"
#include "flowunits/SineOscillator.h"
#include "flowunits/SawtoothOscillator.h"
#include "flowunits/TriangleOscillator.h"
#include "flowunits/WhiteNoise.h"
#include "FullDuplexAnalyzer.h"
#include "FullDuplexEcho.h"
#include "FullDuplexGlitches.h"
#include "FullDuplexLatency.h"
#include "FullDuplexStream.h"
#include "analyzer/GlitchAnalyzer.h"
#include "analyzer/DataPathAnalyzer.h"
#include "InputStreamCallbackAnalyzer.h"
#include "MultiChannelRecording.h"
#include "OboeStreamCallbackProxy.h"
#include "OboeTools.h"
#include "PlayRecordingCallback.h"
#include "SawPingGenerator.h"
#include "flowunits/TriangleOscillator.h"
// These must match order in strings.xml and in StreamConfiguration.java
#define NATIVE_MODE_UNSPECIFIED 0
#define NATIVE_MODE_OPENSLES 1
#define NATIVE_MODE_AAUDIO 2
#define MAX_SINE_OSCILLATORS 8
#define MAX_SINE_OSCILLATORS 16
#define AMPLITUDE_SINE 1.0
#define AMPLITUDE_SAWTOOTH 0.5
#define FREQUENCY_SAW_PING 800.0
#define AMPLITUDE_SAW_PING 0.8
#define AMPLITUDE_IMPULSE 0.7
#define NANOS_PER_MICROSECOND ((int64_t) 1000)
#define NANOS_PER_MILLISECOND (1000 * NANOS_PER_MICROSECOND)
#define NANOS_PER_SECOND (1000 * NANOS_PER_MILLISECOND)
#define LIB_AAUDIO_NAME "libaaudio.so"
#define FUNCTION_IS_MMAP "AAudioStream_isMMapUsed"
#define FUNCTION_SET_MMAP_POLICY "AAudio_setMMapPolicy"
#define FUNCTION_GET_MMAP_POLICY "AAudio_getMMapPolicy"
#define SECONDS_TO_RECORD 10
typedef struct AAudioStreamStruct AAudioStream;
/**
* Call some AAudio test routines that are not part of the normal API.
*/
class AAudioExtensions {
public:
AAudioExtensions() {
int32_t policy = getIntegerProperty("aaudio.mmap_policy", 0);
mMMapSupported = isPolicyEnabled(policy);
policy = getIntegerProperty("aaudio.mmap_exclusive_policy", 0);
mMMapExclusiveSupported = isPolicyEnabled(policy);
}
static bool isPolicyEnabled(int32_t policy) {
return (policy == AAUDIO_POLICY_AUTO || policy == AAUDIO_POLICY_ALWAYS);
}
static AAudioExtensions &getInstance() {
static AAudioExtensions instance;
return instance;
}
bool isMMapUsed(oboe::AudioStream *oboeStream) {
if (!loadLibrary()) return false;
AAudioStream *aaudioStream = (AAudioStream *) oboeStream->getUnderlyingStream();
return mAAudioStream_isMMap(aaudioStream);
}
bool setMMapEnabled(bool enabled) {
if (!loadLibrary()) return false;
return mAAudio_setMMapPolicy(enabled ? AAUDIO_POLICY_AUTO : AAUDIO_POLICY_NEVER);
}
bool isMMapEnabled() {
if (!loadLibrary()) return false;
int32_t policy = mAAudio_getMMapPolicy();
return isPolicyEnabled(policy);
}
bool isMMapSupported() {
return mMMapSupported;
}
bool isMMapExclusiveSupported() {
return mMMapExclusiveSupported;
}
private:
enum {
AAUDIO_POLICY_NEVER = 1,
AAUDIO_POLICY_AUTO,
AAUDIO_POLICY_ALWAYS
};
typedef int32_t aaudio_policy_t;
int getIntegerProperty(const char *name, int defaultValue) {
int result = defaultValue;
char valueText[PROP_VALUE_MAX] = {0};
if (__system_property_get(name, valueText) != 0) {
result = atoi(valueText);
}
return result;
}
// return true if it succeeds
bool loadLibrary() {
if (mFirstTime) {
mFirstTime = false;
mLibHandle = dlopen(LIB_AAUDIO_NAME, 0);
if (mLibHandle == nullptr) {
LOGI("%s() could not find " LIB_AAUDIO_NAME, __func__);
return false;
}
mAAudioStream_isMMap = (bool (*)(AAudioStream *stream))
dlsym(mLibHandle, FUNCTION_IS_MMAP);
if (mAAudioStream_isMMap == nullptr) {
LOGI("%s() could not find " FUNCTION_IS_MMAP, __func__);
return false;
}
mAAudio_setMMapPolicy = (int32_t (*)(aaudio_policy_t policy))
dlsym(mLibHandle, FUNCTION_SET_MMAP_POLICY);
if (mAAudio_setMMapPolicy == nullptr) {
LOGI("%s() could not find " FUNCTION_SET_MMAP_POLICY, __func__);
return false;
}
mAAudio_getMMapPolicy = (aaudio_policy_t (*)())
dlsym(mLibHandle, FUNCTION_GET_MMAP_POLICY);
if (mAAudio_getMMapPolicy == nullptr) {
LOGI("%s() could not find " FUNCTION_GET_MMAP_POLICY, __func__);
return false;
}
}
return true;
}
bool mFirstTime = true;
void *mLibHandle = nullptr;
bool (*mAAudioStream_isMMap)(AAudioStream *stream) = nullptr;
int32_t (*mAAudio_setMMapPolicy)(aaudio_policy_t policy) = nullptr;
aaudio_policy_t (*mAAudio_getMMapPolicy)() = nullptr;
bool mMMapSupported = false;
bool mMMapExclusiveSupported = false;
};
/**
* Abstract base class that corresponds to a test at the Java level.
*/
@@ -189,12 +77,13 @@ class ActivityContext {
public:
ActivityContext() {}
virtual ~ActivityContext() = default;
oboe::AudioStream *getStream(int32_t streamIndex) {
std::shared_ptr<oboe::AudioStream> getStream(int32_t streamIndex) {
auto it = mOboeStreams.find(streamIndex);
if (it != mOboeStreams.end()) {
return it->second.get();
return it->second;
} else {
return nullptr;
}
@@ -202,52 +91,87 @@ public:
virtual void configureBuilder(bool isInput, oboe::AudioStreamBuilder &builder);
/**
* Open a stream with the given parameters.
* @param nativeApi
* @param sampleRate
* @param channelCount
* @param channelMask
* @param format
* @param sharingMode
* @param performanceMode
* @param inputPreset
* @param deviceId
* @param sessionId
* @param framesPerBurst
* @param channelConversionAllowed
* @param formatConversionAllowed
* @param rateConversionQuality
* @param isMMap
* @param isInput
* @return stream ID
*/
int open(jint nativeApi,
jint sampleRate,
jint channelCount,
jint channelMask,
jint format,
jint sharingMode,
jint performanceMode,
jint inputPreset,
jint usage,
jint contentType,
jint bufferCapacityInFrames,
jint deviceId,
jint sessionId,
jint framesPerBurst,
jboolean channelConversionAllowed,
jboolean formatConversionAllowed,
jint rateConversionQuality,
jboolean isMMap,
jboolean isInput);
oboe::Result release();
virtual void close(int32_t streamIndex);
void printScheduler() {
#if OBOE_ENABLE_LOGGING
int scheduler = audioStreamGateway.getScheduler();
#endif
LOGI("scheduler = 0x%08x, SCHED_FIFO = 0x%08X\n", scheduler, SCHED_FIFO);
}
virtual void configureForStart() {}
virtual void configureAfterOpen() {}
oboe::Result start();
oboe::Result pause();
oboe::Result flush();
oboe::Result stopAllStreams();
virtual oboe::Result stop() {
return stopAllStreams();
}
double getCpuLoad() {
float getCpuLoad() {
return oboeCallbackProxy.getCpuLoad();
}
void setWorkload(double workload) {
float getAndResetMaxCpuLoad() {
return oboeCallbackProxy.getAndResetMaxCpuLoad();
}
uint32_t getAndResetCpuMask() {
return oboeCallbackProxy.getAndResetCpuMask();
}
std::string getCallbackTimeString() {
return oboeCallbackProxy.getCallbackTimeString();
}
void setWorkload(int32_t workload) {
oboeCallbackProxy.setWorkload(workload);
}
void setHearWorkload(bool enabled) {
oboeCallbackProxy.setHearWorkload(enabled);
}
virtual oboe::Result startPlayback() {
return oboe::Result::OK;
}
@@ -275,9 +199,66 @@ public:
return 0.0;
}
virtual void setEnabled(bool enabled) {
static int64_t getNanoseconds(clockid_t clockId = CLOCK_MONOTONIC) {
struct timespec time;
int result = clock_gettime(clockId, &time);
if (result < 0) {
return result;
}
return (time.tv_sec * NANOS_PER_SECOND) + time.tv_nsec;
}
// Calculate time between beginning and when frame[0] occurred.
int32_t calculateColdStartLatencyMillis(int32_t sampleRate,
int64_t beginTimeNanos,
int64_t timeStampPosition,
int64_t timestampNanos) const {
int64_t elapsedNanos = NANOS_PER_SECOND * (timeStampPosition / (double) sampleRate);
int64_t timeOfFrameZero = timestampNanos - elapsedNanos;
int64_t coldStartLatencyNanos = timeOfFrameZero - beginTimeNanos;
return coldStartLatencyNanos / NANOS_PER_MILLISECOND;
}
int32_t getColdStartInputMillis() {
std::shared_ptr<oboe::AudioStream> oboeStream = getInputStream();
if (oboeStream != nullptr) {
int64_t framesRead = oboeStream->getFramesRead();
if (framesRead > 0) {
// Base latency on the time that frame[0] would have been received by the app.
int64_t nowNanos = getNanoseconds();
return calculateColdStartLatencyMillis(oboeStream->getSampleRate(),
mInputOpenedAt,
framesRead,
nowNanos);
}
}
return -1;
}
int32_t getColdStartOutputMillis() {
std::shared_ptr<oboe::AudioStream> oboeStream = getOutputStream();
if (oboeStream != nullptr) {
auto result = oboeStream->getTimestamp(CLOCK_MONOTONIC);
if (result) {
auto frameTimestamp = result.value();
// Calculate the time that frame[0] would have been played by the speaker.
int64_t position = frameTimestamp.position;
int64_t timestampNanos = frameTimestamp.timestamp;
return calculateColdStartLatencyMillis(oboeStream->getSampleRate(),
mOutputOpenedAt,
position,
timestampNanos);
}
}
return -1;
}
/**
* Trigger a sound or impulse.
* @param enabled
*/
virtual void trigger() {}
bool isMMapUsed(int32_t streamIndex);
int32_t getFramesPerBlock() {
@@ -288,6 +269,14 @@ public:
return oboeCallbackProxy.getCallbackCount();
}
oboe::Result getLastErrorCallbackResult() {
std::shared_ptr<oboe::AudioStream> stream = getOutputStream();
if (stream == nullptr) {
stream = getInputStream();
}
return stream ? oboe::Result::ErrorNull : stream->getLastErrorCallbackResult();
}
int32_t getFramesPerCallback() {
return oboeCallbackProxy.getFramesPerCallback();
}
@@ -296,6 +285,8 @@ public:
virtual void setSignalType(int signalType) {}
virtual void setAmplitude(float amplitude) {}
virtual int32_t saveWaveFile(const char *filename);
virtual void setMinimumFramesBeforeRead(int32_t numFrames) {}
@@ -303,9 +294,19 @@ public:
static bool mUseCallback;
static int callbackSize;
double getTimestampLatency(int32_t streamIndex);
void setCpuAffinityMask(uint32_t mask) {
oboeCallbackProxy.setCpuAffinityMask(mask);
}
void setWorkloadReportingEnabled(bool enabled) {
oboeCallbackProxy.setWorkloadReportingEnabled(enabled);
}
protected:
oboe::AudioStream *getInputStream();
oboe::AudioStream *getOutputStream();
std::shared_ptr<oboe::AudioStream> getInputStream();
std::shared_ptr<oboe::AudioStream> getOutputStream();
int32_t allocateStreamIndex();
void freeStreamIndex(int32_t streamIndex);
@@ -314,7 +315,7 @@ protected:
SECONDS_TO_RECORD * mSampleRate);
}
virtual void finishOpen(bool isInput, oboe::AudioStream *oboeStream) {}
virtual void finishOpen(bool isInput, std::shared_ptr<oboe::AudioStream> &oboeStream) {}
virtual oboe::Result startStreams() = 0;
@@ -332,9 +333,11 @@ protected:
int32_t mSampleRate = 0; // TODO per stream
std::atomic<bool> threadEnabled{false};
std::thread *dataThread = nullptr;
std::thread *dataThread = nullptr; // FIXME never gets deleted
private:
int64_t mInputOpenedAt = 0;
int64_t mOutputOpenedAt = 0;
};
/**
@@ -346,7 +349,7 @@ public:
ActivityTestInput() {}
virtual ~ActivityTestInput() = default;
void configureForStart() override;
void configureAfterOpen() override;
double getPeakLevel(int index) override {
return mInputAnalyzer.getPeakLevel(index);
@@ -354,8 +357,6 @@ public:
void runBlockingIO() override;
InputStreamCallbackAnalyzer mInputAnalyzer;
void setMinimumFramesBeforeRead(int32_t numFrames) override {
mInputAnalyzer.setMinimumFramesBeforeRead(numFrames);
mMinimumFramesBeforeRead = numFrames;
@@ -369,9 +370,13 @@ protected:
oboe::Result startStreams() override {
mInputAnalyzer.reset();
mInputAnalyzer.setup(std::max(getInputStream()->getFramesPerBurst(), callbackSize),
getInputStream()->getChannelCount(),
getInputStream()->getFormat());
return getInputStream()->requestStart();
}
InputStreamCallbackAnalyzer mInputAnalyzer;
int32_t mMinimumFramesBeforeRead = 0;
};
@@ -414,11 +419,9 @@ public:
void close(int32_t streamIndex) override;
oboe::Result startStreams() override {
return getOutputStream()->start();
}
oboe::Result startStreams() override;
void configureForStart() override;
void configureAfterOpen() override;
virtual void configureStreamGateway();
@@ -439,6 +442,13 @@ public:
mSignalType = (SignalType) signalType;
}
void setAmplitude(float amplitude) override {
mAmplitude = amplitude;
if (mVolumeRamp) {
mVolumeRamp->setTarget(mAmplitude);
}
}
protected:
SignalType mSignalType = SignalType::Sine;
@@ -446,15 +456,22 @@ protected:
std::vector<SawtoothOscillator> sawtoothOscillators;
static constexpr float kSweepPeriod = 10.0; // for triangle up and down
// A triangle LFO is shaped into either a linear or an exponential range.
// A triangle LFO is shaped into either a linear or an exponential range for sweep.
TriangleOscillator mTriangleOscillator;
LinearShape mLinearShape;
ExponentialShape mExponentialShape;
class WhiteNoise mWhiteNoise;
static constexpr int kRampMSec = 10; // for volume control
float mAmplitude = 1.0f;
std::shared_ptr<RampLinear> mVolumeRamp;
std::unique_ptr<ManyToMultiConverter> manyToMulti;
std::unique_ptr<MonoToMultiConverter> monoToMulti;
std::shared_ptr<flowgraph::SinkFloat> mSinkFloat;
std::shared_ptr<flowgraph::SinkI16> mSinkI16;
std::shared_ptr<oboe::flowgraph::SinkFloat> mSinkFloat;
std::shared_ptr<oboe::flowgraph::SinkI16> mSinkI16;
std::shared_ptr<oboe::flowgraph::SinkI24> mSinkI24;
std::shared_ptr<oboe::flowgraph::SinkI32> mSinkI32;
};
/**
@@ -466,10 +483,10 @@ public:
ActivityTapToTone() {}
virtual ~ActivityTapToTone() = default;
void configureForStart() override;
void configureAfterOpen() override;
virtual void setEnabled(bool enabled) override {
sawPingGenerator.setEnabled(enabled);
virtual void trigger() override {
sawPingGenerator.trigger();
}
SawPingGenerator sawPingGenerator;
@@ -522,12 +539,16 @@ public:
}
}
double getPeakLevel(int index) override {
return mFullDuplexEcho->getPeakLevel(index);
}
FullDuplexAnalyzer *getFullDuplexAnalyzer() override {
return (FullDuplexAnalyzer *) mFullDuplexEcho.get();
}
protected:
void finishOpen(bool isInput, oboe::AudioStream *oboeStream) override;
void finishOpen(bool isInput, std::shared_ptr<oboe::AudioStream> &oboeStream) override;
private:
std::unique_ptr<FullDuplexEcho> mFullDuplexEcho{};
@@ -538,36 +559,74 @@ private:
*/
class ActivityRoundTripLatency : public ActivityFullDuplex {
public:
ActivityRoundTripLatency() {
#define USE_WHITE_NOISE_ANALYZER 1
#if USE_WHITE_NOISE_ANALYZER
// New analyzer that uses a short pattern of white noise bursts.
mLatencyAnalyzer = std::make_unique<WhiteNoiseLatencyAnalyzer>();
#else
// Old analyzer based on encoded random bits.
mLatencyAnalyzer = std::make_unique<EncodedRandomLatencyAnalyzer>();
#endif
mLatencyAnalyzer->setup();
}
virtual ~ActivityRoundTripLatency() = default;
oboe::Result startStreams() override {
mAnalyzerLaunched = false;
return mFullDuplexLatency->start();
}
void configureBuilder(bool isInput, oboe::AudioStreamBuilder &builder) override;
LatencyAnalyzer *getLatencyAnalyzer() {
return mFullDuplexLatency->getLatencyAnalyzer();
return mLatencyAnalyzer.get();
}
int32_t getState() override {
return getLatencyAnalyzer()->getState();
}
int32_t getResult() override {
return getLatencyAnalyzer()->getState();
return getLatencyAnalyzer()->getState(); // TODO This does not look right.
}
bool isAnalyzerDone() override {
return mFullDuplexLatency->isDone();
if (!mAnalyzerLaunched) {
mAnalyzerLaunched = launchAnalysisIfReady();
}
return mLatencyAnalyzer->isDone();
}
FullDuplexAnalyzer *getFullDuplexAnalyzer() override {
return (FullDuplexAnalyzer *) mFullDuplexLatency.get();
}
static void analyzeData(LatencyAnalyzer *analyzer) {
analyzer->analyze();
}
bool launchAnalysisIfReady() {
// Are we ready to do the analysis?
if (mLatencyAnalyzer->hasEnoughData()) {
// Crunch the numbers on a separate thread.
std::thread t(analyzeData, mLatencyAnalyzer.get());
t.detach();
return true;
}
return false;
}
jdouble measureTimestampLatency();
protected:
void finishOpen(bool isInput, oboe::AudioStream *oboeStream) override;
void finishOpen(bool isInput, std::shared_ptr<oboe::AudioStream> &oboeStream) override;
private:
std::unique_ptr<FullDuplexLatency> mFullDuplexLatency{};
std::unique_ptr<FullDuplexAnalyzer> mFullDuplexLatency{};
std::unique_ptr<LatencyAnalyzer> mLatencyAnalyzer;
bool mAnalyzerLaunched = false;
};
/**
@@ -583,18 +642,19 @@ public:
void configureBuilder(bool isInput, oboe::AudioStreamBuilder &builder) override;
GlitchAnalyzer *getGlitchAnalyzer() {
if (!mFullDuplexGlitches) return nullptr;
return mFullDuplexGlitches->getGlitchAnalyzer();
return &mGlitchAnalyzer;
}
int32_t getState() override {
return getGlitchAnalyzer()->getState();
}
int32_t getResult() override {
return getGlitchAnalyzer()->getResult();
}
bool isAnalyzerDone() override {
return mFullDuplexGlitches->isDone();
return mGlitchAnalyzer.isDone();
}
FullDuplexAnalyzer *getFullDuplexAnalyzer() override {
@@ -602,10 +662,54 @@ public:
}
protected:
void finishOpen(bool isInput, oboe::AudioStream *oboeStream) override;
void finishOpen(bool isInput, std::shared_ptr<oboe::AudioStream> &oboeStream) override;
private:
std::unique_ptr<FullDuplexGlitches> mFullDuplexGlitches{};
std::unique_ptr<FullDuplexAnalyzer> mFullDuplexGlitches{};
GlitchAnalyzer mGlitchAnalyzer;
};
/**
* Measure Data Path
*/
class ActivityDataPath : public ActivityFullDuplex {
public:
oboe::Result startStreams() override {
return mFullDuplexDataPath->start();
}
void configureBuilder(bool isInput, oboe::AudioStreamBuilder &builder) override;
void configureAfterOpen() override {
// set buffer size
std::shared_ptr<oboe::AudioStream> outputStream = getOutputStream();
int32_t capacityInFrames = outputStream->getBufferCapacityInFrames();
int32_t burstInFrames = outputStream->getFramesPerBurst();
int32_t capacityInBursts = capacityInFrames / burstInFrames;
int32_t sizeInBursts = std::max(2, capacityInBursts / 2);
// Set size of buffer to minimize underruns.
auto result = outputStream->setBufferSizeInFrames(sizeInBursts * burstInFrames);
static_cast<void>(result); // Avoid unused variable.
LOGD("ActivityDataPath: %s() capacity = %d, burst = %d, size = %d",
__func__, capacityInFrames, burstInFrames, result.value());
}
DataPathAnalyzer *getDataPathAnalyzer() {
return &mDataPathAnalyzer;
}
FullDuplexAnalyzer *getFullDuplexAnalyzer() override {
return (FullDuplexAnalyzer *) mFullDuplexDataPath.get();
}
protected:
void finishOpen(bool isInput, std::shared_ptr<oboe::AudioStream> &oboeStream) override;
private:
std::unique_ptr<FullDuplexAnalyzer> mFullDuplexDataPath{};
DataPathAnalyzer mDataPathAnalyzer;
};
/**
@@ -620,28 +724,29 @@ public:
void close(int32_t streamIndex) override;
oboe::Result startStreams() override {
oboe::AudioStream *outputStream = getOutputStream();
std::shared_ptr<oboe::AudioStream> outputStream = getOutputStream();
if (outputStream) {
return outputStream->start();
}
oboe::AudioStream *inputStream = getInputStream();
std::shared_ptr<oboe::AudioStream> inputStream = getInputStream();
if (inputStream) {
return inputStream->start();
}
return oboe::Result::ErrorNull;
}
void configureForStart() override;
void configureAfterOpen() override;
private:
std::unique_ptr<SineOscillator> sineOscillator;
std::unique_ptr<MonoToMultiConverter> monoToMulti;
std::shared_ptr<flowgraph::SinkFloat> mSinkFloat;
std::shared_ptr<oboe::flowgraph::SinkFloat> mSinkFloat;
};
/**
* Switch between various
* Global context for native tests.
* Switch between various ActivityContexts.
*/
class NativeAudioContext {
public:
@@ -679,6 +784,9 @@ public:
case ActivityType::TestDisconnect:
currentActivity = &mActivityTestDisconnect;
break;
case ActivityType::DataPath:
currentActivity = &mActivityDataPath;
break;
}
}
@@ -693,6 +801,7 @@ public:
ActivityEcho mActivityEcho;
ActivityRoundTripLatency mActivityRoundTripLatency;
ActivityGlitches mActivityGlitches;
ActivityDataPath mActivityDataPath;
ActivityTestDisconnect mActivityTestDisconnect;
private:
@@ -708,11 +817,11 @@ private:
RoundTripLatency = 5,
Glitches = 6,
TestDisconnect = 7,
DataPath = 8,
};
ActivityType mActivityType = ActivityType::Undefined;
ActivityContext *currentActivity = &mActivityTestOutput;
};
#endif //NATIVEOBOE_NATIVEAUDIOCONTEXT_H
@@ -17,40 +17,8 @@
#include "common/OboeDebug.h"
#include "OboeStreamCallbackProxy.h"
// Linear congruential random number generator.
static uint32_t s_random16() {
static uint32_t seed = 1234;
seed = ((seed * 31421) + 6927) & 0x0FFFF;
return seed;
}
/**
* The random number generator is good for burning CPU because the compiler cannot
* easily optimize away the computation.
* @param workload number of times to execute the loop
* @return a white noise value between -1.0 and +1.0
*/
static float s_burnCPU(int32_t workload) {
uint32_t random = 0;
for (int32_t i = 0; i < workload; i++) {
for (int32_t j = 0; j < 10; j++) {
random = random ^ s_random16();
}
}
return (random - 32768) * (1.0 / 32768);
}
bool OboeStreamCallbackProxy::mCallbackReturnStop = false;
int64_t OboeStreamCallbackProxy::getNanoseconds(clockid_t clockId) {
struct timespec time;
int result = clock_gettime(clockId, &time);
if (result < 0) {
return result;
}
return (time.tv_sec * 1e9) + time.tv_nsec;
}
oboe::DataCallbackResult OboeStreamCallbackProxy::onAudioReady(
oboe::AudioStream *audioStream,
void *audioData,
@@ -58,6 +26,23 @@ oboe::DataCallbackResult OboeStreamCallbackProxy::onAudioReady(
oboe::DataCallbackResult callbackResult = oboe::DataCallbackResult::Stop;
int64_t startTimeNanos = getNanoseconds();
// Record which CPU this is running on.
orCurrentCpuMask(sched_getcpu());
// Tell ADPF in advance what our workload will be.
if (mWorkloadReportingEnabled) {
audioStream->reportWorkload(mNumWorkloadVoices);
}
// Change affinity if app requested a change.
uint32_t mask = mCpuAffinityMask;
if (mask != mPreviousMask) {
int err = applyCpuAffinityMask(mask);
if (err != 0) {
}
mPreviousMask = mask;
}
mCallbackCount++;
mFramesPerCallback = numFrames;
@@ -65,31 +50,67 @@ oboe::DataCallbackResult OboeStreamCallbackProxy::onAudioReady(
return oboe::DataCallbackResult::Stop;
}
s_burnCPU((int32_t)(mWorkload * kWorkloadScaler * numFrames));
if (mCallback != nullptr) {
callbackResult = mCallback->onAudioReady(audioStream, audioData, numFrames);
}
// Update CPU load
double calculationTime = (double)(getNanoseconds() - startTimeNanos);
double inverseRealTime = audioStream->getSampleRate() / (1.0e9 * numFrames);
double currentCpuLoad = calculationTime * inverseRealTime; // avoid a divide
mCpuLoad = (mCpuLoad * 0.95) + (currentCpuLoad * 0.05); // simple low pass filter
mSynthWorkload.onCallback(mNumWorkloadVoices);
if (mNumWorkloadVoices > 0) {
// Render into the buffer or discard the synth voices.
float *buffer = (audioStream->getChannelCount() == 2 && mHearWorkload)
? static_cast<float *>(audioData) : nullptr;
mSynthWorkload.renderStereo(buffer, numFrames);
}
// Measure CPU load.
int64_t currentTimeNanos = getNanoseconds();
// Sometimes we get a short callback when doing sample rate conversion.
// Just ignore those to avoid noise.
if (numFrames > (getFramesPerCallback() / 2)) {
int64_t calculationTime = currentTimeNanos - startTimeNanos;
float currentCpuLoad = calculationTime * 0.000000001f * audioStream->getSampleRate() / numFrames;
mCpuLoad = (mCpuLoad * 0.95f) + (currentCpuLoad * 0.05f); // simple low pass filter
mMaxCpuLoad = std::max(currentCpuLoad, mMaxCpuLoad.load());
}
if (mPreviousCallbackTimeNs != 0) {
mStatistics.add((currentTimeNanos - mPreviousCallbackTimeNs) * kNsToMsScaler);
}
mPreviousCallbackTimeNs = currentTimeNanos;
return callbackResult;
}
void OboeStreamCallbackProxy::onErrorBeforeClose(oboe::AudioStream *audioStream, oboe::Result error) {
LOGD("OboeStreamCallbackProxy::%s(%p, %d) called", __func__, audioStream, error);
if (mCallback != nullptr) {
mCallback->onErrorBeforeClose(audioStream, error);
int OboeStreamCallbackProxy::applyCpuAffinityMask(uint32_t mask) {
int err = 0;
// Capture original CPU set so we can restore it.
if (!mIsOriginalCpuSetValid) {
err = sched_getaffinity((pid_t) 0,
sizeof(mOriginalCpuSet),
&mOriginalCpuSet);
if (err) {
LOGE("%s(0x%02X) - sched_getaffinity(), errno = %d\n", __func__, mask, errno);
return -errno;
}
mIsOriginalCpuSetValid = true;
}
}
void OboeStreamCallbackProxy::onErrorAfterClose(oboe::AudioStream *audioStream, oboe::Result error) {
LOGD("OboeStreamCallbackProxy::%s(%p, %d) called", __func__, audioStream, error);
if (mCallback != nullptr) {
mCallback->onErrorAfterClose(audioStream, error);
if (mask) {
cpu_set_t cpu_set;
CPU_ZERO(&cpu_set);
int cpuCount = sysconf(_SC_NPROCESSORS_CONF);
for (int cpuIndex = 0; cpuIndex < cpuCount; cpuIndex++) {
if (mask & (1 << cpuIndex)) {
CPU_SET(cpuIndex, &cpu_set);
}
}
err = sched_setaffinity((pid_t) 0, sizeof(cpu_set_t), &cpu_set);
} else {
// Restore original mask.
err = sched_setaffinity((pid_t) 0, sizeof(mOriginalCpuSet), &mOriginalCpuSet);
}
if (err) {
LOGE("%s(0x%02X) - sched_setaffinity(), errno = %d\n", __func__, mask, errno);
return -errno;
}
return 0;
}
@@ -19,15 +19,129 @@
#include <unistd.h>
#include <sys/types.h>
#include <sys/sysinfo.h>
#include "oboe/Oboe.h"
#include "synth/Synthesizer.h"
#include "synth/SynthTools.h"
#include "OboeTesterStreamCallback.h"
class OboeStreamCallbackProxy : public oboe::AudioStreamCallback {
class DoubleStatistics {
public:
void add(double statistic) {
if (skipCount < kNumberStatisticsToSkip) {
skipCount++;
} else {
if (statistic <= 0.0) return;
sum = statistic + sum;
count++;
minimum = std::min(statistic, minimum.load());
maximum = std::max(statistic, maximum.load());
}
}
double getAverage() const {
return sum / count;
}
std::string dump() const {
if (count == 0) return "?";
char buff[100];
snprintf(buff, sizeof(buff), "%3.1f/%3.1f/%3.1f ms", minimum.load(), getAverage(), maximum.load());
std::string buffAsStr = buff;
return buffAsStr;
}
void clear() {
skipCount = 0;
sum = 0;
count = 0;
minimum = DBL_MAX;
maximum = 0;
}
private:
static constexpr double kNumberStatisticsToSkip = 5; // Skip the first 5 frames
std::atomic<int> skipCount { 0 };
std::atomic<double> sum { 0 };
std::atomic<int> count { 0 };
std::atomic<double> minimum { DBL_MAX };
std::atomic<double> maximum { 0 };
};
/**
* Manage the synthesizer workload that burdens the CPU.
* Adjust the number of voices according to the requested workload.
* Trigger noteOn and noteOff messages.
*/
class SynthWorkload {
public:
SynthWorkload() {
mSynth.setup(marksynth::kSynthmarkSampleRate, marksynth::kSynthmarkMaxVoices);
}
void onCallback(double workload) {
// If workload changes then restart notes.
if (workload != mPreviousWorkload) {
mSynth.allNotesOff();
mAreNotesOn = false;
mCountdown = 0; // trigger notes on
mPreviousWorkload = workload;
}
if (mCountdown <= 0) {
if (mAreNotesOn) {
mSynth.allNotesOff();
mAreNotesOn = false;
mCountdown = mOffFrames;
} else {
mSynth.notesOn((int)mPreviousWorkload);
mAreNotesOn = true;
mCountdown = mOnFrames;
}
}
}
/**
* Render the notes into a stereo buffer.
* Passing a nullptr will cause the calculated results to be discarded.
* The workload should be the same.
* @param buffer a real stereo buffer or nullptr
* @param numFrames
*/
void renderStereo(float *buffer, int numFrames) {
if (buffer == nullptr) {
int framesLeft = numFrames;
while (framesLeft > 0) {
int framesThisTime = std::min(kDummyBufferSizeInFrames, framesLeft);
// Do the work then throw it away.
mSynth.renderStereo(&mDummyStereoBuffer[0], framesThisTime);
framesLeft -= framesThisTime;
}
} else {
mSynth.renderStereo(buffer, numFrames);
}
mCountdown -= numFrames;
}
private:
marksynth::Synthesizer mSynth;
static constexpr int kDummyBufferSizeInFrames = 32;
float mDummyStereoBuffer[kDummyBufferSizeInFrames * 2];
double mPreviousWorkload = 1.0;
bool mAreNotesOn = false;
int mCountdown = 0;
int mOnFrames = (int) (0.2 * 48000);
int mOffFrames = (int) (0.3 * 48000);
};
class OboeStreamCallbackProxy : public OboeTesterStreamCallback {
public:
void setCallback(oboe::AudioStreamCallback *callback) {
void setDataCallback(oboe::AudioStreamDataCallback *callback) {
mCallback = callback;
setCallbackCount(0);
mStatistics.clear();
mPreviousMask = 0;
}
static void setCallbackReturnStop(bool b) {
@@ -54,39 +168,106 @@ public:
void *audioData,
int numFrames) override;
void onErrorBeforeClose(oboe::AudioStream *audioStream, oboe::Result error) override;
void onErrorAfterClose(oboe::AudioStream *audioStream, oboe::Result error) override;
/**
* Specify the amount of artificial workload that will waste CPU cycles
* and increase the CPU load.
* @param workload typically ranges from 0.0 to 100.0
* @param workload typically ranges from 0 to 400
*/
void setWorkload(double workload) {
mWorkload = std::max(0.0, workload);
void setWorkload(int32_t workload) {
mNumWorkloadVoices = std::max(0, workload);
}
double getWorkload() const {
return mWorkload;
int32_t getWorkload() const {
return mNumWorkloadVoices;
}
double getCpuLoad() const {
void setHearWorkload(bool enabled) {
mHearWorkload = enabled;
}
/**
* This is the callback duration relative to the real-time equivalent.
* So it may be higher than 1.0.
* @return low pass filtered value for the fractional CPU load
*/
float getCpuLoad() const {
return mCpuLoad;
}
static int64_t getNanoseconds(clockid_t clockId = CLOCK_MONOTONIC);
/**
* Calling this will atomically reset the max to zero so only call
* this from one client.
*
* @return last value of the maximum unfiltered CPU load.
*/
float getAndResetMaxCpuLoad() {
return mMaxCpuLoad.exchange(0.0f);
}
std::string getCallbackTimeString() const {
return mStatistics.dump();
}
/**
* @return mask of the CPUs used since the last reset
*/
uint32_t getAndResetCpuMask() {
return mCpuMask.exchange(0);
}
void orCurrentCpuMask(int cpuIndex) {
mCpuMask |= (1 << cpuIndex);
}
/**
* @param cpuIndex
* @return 0 on success or a negative errno
*/
int setCpuAffinity(int cpuIndex) {
cpu_set_t cpu_set;
CPU_ZERO(&cpu_set);
CPU_SET(cpuIndex, &cpu_set);
int err = sched_setaffinity((pid_t) 0, sizeof(cpu_set_t), &cpu_set);
return err == 0 ? 0 : -errno;
}
/**
*
* @param mask bits for each CPU or zero for all
* @return
*/
int applyCpuAffinityMask(uint32_t mask);
void setCpuAffinityMask(uint32_t mask) {
mCpuAffinityMask = mask;
}
void setWorkloadReportingEnabled(bool enabled) {
mWorkloadReportingEnabled = enabled;
}
private:
static constexpr int32_t kWorkloadScaler = 500;
double mWorkload = 0.0;
std::atomic<double> mCpuLoad{0};
static constexpr double kNsToMsScaler = 0.000001;
std::atomic<float> mCpuLoad{0.0f};
std::atomic<float> mMaxCpuLoad{0.0f};
int64_t mPreviousCallbackTimeNs = 0;
DoubleStatistics mStatistics;
int32_t mNumWorkloadVoices = 0;
SynthWorkload mSynthWorkload;
bool mHearWorkload = false;
bool mWorkloadReportingEnabled = false;
oboe::AudioStreamCallback *mCallback = nullptr;
oboe::AudioStreamDataCallback *mCallback = nullptr;
static bool mCallbackReturnStop;
int64_t mCallbackCount = 0;
std::atomic<int32_t> mFramesPerCallback{0};
std::atomic<uint32_t> mCpuAffinityMask{0};
std::atomic<uint32_t> mPreviousMask{0};
std::atomic<uint32_t> mCpuMask{0};
cpu_set_t mOriginalCpuSet;
bool mIsOriginalCpuSetValid = false;
};
#endif //NATIVEOBOE_OBOESTREAMCALLBACKPROXY_H
@@ -0,0 +1,86 @@
/*
* Copyright 2020 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#include <sched.h>
#include <cstring>
#include "AudioStreamGateway.h"
#include "common/OboeDebug.h"
#include "oboe/Oboe.h"
#include "OboeStreamCallbackProxy.h"
#include "OboeTesterStreamCallback.h"
#include "OboeTools.h"
#include "synth/IncludeMeOnce.h"
int32_t OboeTesterStreamCallback::mHangTimeMillis = 0;
// Print if scheduler changes.
void OboeTesterStreamCallback::printScheduler() {
#if OBOE_ENABLE_LOGGING
int scheduler = sched_getscheduler(gettid());
if (scheduler != mPreviousScheduler) {
int schedulerType = scheduler & 0xFFFF; // mask off high flags
LOGD("callback CPU scheduler = 0x%08x = %s",
scheduler,
((schedulerType == SCHED_FIFO) ? "SCHED_FIFO" :
((schedulerType == SCHED_OTHER) ? "SCHED_OTHER" :
((schedulerType == SCHED_RR) ? "SCHED_RR" : "UNKNOWN")))
);
mPreviousScheduler = scheduler;
}
#endif
}
// Sleep to cause an XRun. Then reschedule.
void OboeTesterStreamCallback::maybeHang(const int64_t startNanos) {
if (mHangTimeMillis == 0) return;
if (startNanos > mNextTimeToHang) {
LOGD("%s() start sleeping", __func__);
// Take short naps until it is time to wake up.
int64_t nowNanos = startNanos;
int64_t wakeupNanos = startNanos + (mHangTimeMillis * NANOS_PER_MILLISECOND);
while (nowNanos < wakeupNanos && mHangTimeMillis > 0) {
int32_t sleepTimeMicros = (int32_t) ((wakeupNanos - nowNanos) / 1000);
if (sleepTimeMicros == 0) break;
// The usleep() function can fail if it sleeps for more than one second.
// So sleep for several small intervals.
// This also allows us to exit the loop if mHangTimeMillis gets set to zero.
const int32_t maxSleepTimeMicros = 100 * 1000;
sleepTimeMicros = std::min(maxSleepTimeMicros, sleepTimeMicros);
usleep(sleepTimeMicros);
nowNanos = getNanoseconds();
}
// Calculate when we hang again.
const int32_t minDurationMillis = 500;
const int32_t maxDurationMillis = std::max(10000, mHangTimeMillis * 2);
int32_t durationMillis = mHangTimeMillis * 10;
durationMillis = std::max(minDurationMillis, std::min(maxDurationMillis, durationMillis));
mNextTimeToHang = startNanos + (durationMillis * NANOS_PER_MILLISECOND);
LOGD("%s() slept for %d msec, durationMillis = %d", __func__,
(int)((nowNanos - startNanos) / 1e6L),
durationMillis);
}
}
int64_t OboeTesterStreamCallback::getNanoseconds(clockid_t clockId) {
struct timespec time;
int result = clock_gettime(clockId, &time);
if (result < 0) {
return result;
}
return (time.tv_sec * 1e9) + time.tv_nsec;
}
@@ -0,0 +1,58 @@
/*
* Copyright 2020 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#ifndef OBOETESTER_STREAM_CALLBACK_H
#define OBOETESTER_STREAM_CALLBACK_H
#include <unistd.h>
#include <sys/types.h>
#include <sys/sysinfo.h>
#include "flowgraph/FlowGraphNode.h"
#include "oboe/Oboe.h"
#include "synth/Synthesizer.h"
#include "synth/SynthTools.h"
class OboeTesterStreamCallback : public oboe::AudioStreamCallback {
public:
virtual ~OboeTesterStreamCallback() = default;
// Call this before starting.
void reset() {
mPreviousScheduler = -1;
}
static int64_t getNanoseconds(clockid_t clockId = CLOCK_MONOTONIC);
/**
* Specify a sleep time that will hang the audio periodically.
*
* @param hangTimeMillis
*/
static void setHangTimeMillis(int hangTimeMillis) {
mHangTimeMillis = hangTimeMillis;
}
protected:
void printScheduler();
void maybeHang(int64_t nowNanos);
int mPreviousScheduler = -1;
static int mHangTimeMillis;
int64_t mNextTimeToHang = 0;
};
#endif //OBOETESTER_STREAM_CALLBACK_H
@@ -1,5 +1,5 @@
/*
* Copyright 2019 The Android Open Source Project
* Copyright 2023 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@@ -14,5 +14,12 @@
* limitations under the License.
*/
#include "common/OboeDebug.h"
#include "FullDuplexGlitches.h"
#ifndef OBOETESTER_OBOETOOLS_H
#define OBOETESTER_OBOETOOLS_H
#define NANOS_PER_MICROSECOND ((int64_t) 1000)
#define NANOS_PER_MILLISECOND (1000 * NANOS_PER_MICROSECOND)
#define NANOS_PER_SECOND (1000 * NANOS_PER_MILLISECOND)
#define MILLISECONDS_PER_SECOND 1000
#endif //OBOETESTER_OBOETOOLS_H
@@ -19,7 +19,7 @@
#include "oboe/Definitions.h"
#include "SawPingGenerator.h"
using namespace flowgraph;
using namespace oboe::flowgraph;
SawPingGenerator::SawPingGenerator()
: OscillatorBase()
@@ -30,6 +30,11 @@ SawPingGenerator::SawPingGenerator()
SawPingGenerator::~SawPingGenerator() { }
void SawPingGenerator::reset() {
FlowGraphNode::reset();
mAcknowledgeCount.store(mRequestCount.load());
}
int32_t SawPingGenerator::onProcess(int numFrames) {
const float *frequencies = frequency.getBuffer();
@@ -58,11 +63,7 @@ int32_t SawPingGenerator::onProcess(int numFrames) {
return numFrames;
}
void SawPingGenerator::setEnabled(bool enabled) {
if (enabled) {
mRequestCount++;
} else {
mAcknowledgeCount.store(mRequestCount.load());
}
void SawPingGenerator::trigger() {
mRequestCount++;
}
@@ -32,7 +32,9 @@ public:
int32_t onProcess(int numFrames) override;
void setEnabled(bool enabled);
void trigger();
void reset() override;
private:
std::atomic<int> mRequestCount; // external thread increments this to request a beep
@@ -0,0 +1,118 @@
/*
* Copyright 2023 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#include <stdlib.h>
#include <aaudio/AAudioExtensions.h>
#include "common/OboeDebug.h"
#include "oboe/AudioClock.h"
#include "TestColdStartLatency.h"
#include "OboeTools.h"
using namespace oboe;
int32_t TestColdStartLatency::open(bool useInput, bool useLowLatency, bool useMmap, bool
useExclusive) {
mDataCallback = std::make_shared<MyDataCallback>();
// Enable MMAP if needed
bool wasMMapEnabled = AAudioExtensions::getInstance().isMMapEnabled();
AAudioExtensions::getInstance().setMMapEnabled(useMmap);
int64_t beginOpenNanos = AudioClock::getNanoseconds();
AudioStreamBuilder builder;
Result result = builder.setFormat(AudioFormat::Float)
->setPerformanceMode(useLowLatency ? PerformanceMode::LowLatency :
PerformanceMode::None)
->setDirection(useInput ? Direction::Input : Direction::Output)
->setChannelCount(kChannelCount)
->setDataCallback(mDataCallback)
->setSharingMode(useExclusive ? SharingMode::Exclusive : SharingMode::Shared)
->openStream(mStream);
int64_t endOpenNanos = AudioClock::getNanoseconds();
int64_t actualDurationNanos = endOpenNanos - beginOpenNanos;
mOpenTimeMicros = actualDurationNanos / NANOS_PER_MICROSECOND;
// Revert MMAP back to its previous state
AAudioExtensions::getInstance().setMMapEnabled(wasMMapEnabled);
mDeviceId = mStream->getDeviceId();
return (int32_t) result;
}
int32_t TestColdStartLatency::start() {
mBeginStartNanos = AudioClock::getNanoseconds();
Result result = mStream->requestStart();
int64_t endStartNanos = AudioClock::getNanoseconds();
int64_t actualDurationNanos = endStartNanos - mBeginStartNanos;
mStartTimeMicros = actualDurationNanos / NANOS_PER_MICROSECOND;
return (int32_t) result;
}
int32_t TestColdStartLatency::close() {
Result result1 = mStream->requestStop();
Result result2 = mStream->close();
return (int32_t)((result1 != Result::OK) ? result1 : result2);
}
int32_t TestColdStartLatency::getColdStartTimeMicros() {
int64_t position;
int64_t timestampNanos;
if (mStream->getDirection() == Direction::Output) {
auto result = mStream->getTimestamp(CLOCK_MONOTONIC);
if (!result) {
return -1; // ERROR
}
auto frameTimestamp = result.value();
// Calculate the time that frame[0] would have been played by the speaker.
position = frameTimestamp.position;
timestampNanos = frameTimestamp.timestamp;
} else {
position = mStream->getFramesRead();
timestampNanos = AudioClock::getNanoseconds();
}
double sampleRate = (double) mStream->getSampleRate();
int64_t elapsedNanos = NANOS_PER_SECOND * (position / sampleRate);
int64_t timeOfFrameZero = timestampNanos - elapsedNanos;
int64_t coldStartLatencyNanos = timeOfFrameZero - mBeginStartNanos;
return coldStartLatencyNanos / NANOS_PER_MICROSECOND;
}
// Callback that sleeps then touches the audio buffer.
DataCallbackResult TestColdStartLatency::MyDataCallback::onAudioReady(
AudioStream *audioStream,
void *audioData,
int32_t numFrames) {
float *floatData = (float *) audioData;
const int numSamples = numFrames * kChannelCount;
if (audioStream->getDirection() == Direction::Output) {
// Fill mono buffer with a sine wave.
for (int i = 0; i < numSamples; i++) {
*floatData++ = sinf(mPhase) * 0.2f;
if ((i % kChannelCount) == (kChannelCount - 1)) {
mPhase += kPhaseIncrement;
// Wrap the phase around in a circle.
if (mPhase >= M_PI) mPhase -= 2 * M_PI;
}
}
}
return DataCallbackResult::Continue;
}
@@ -0,0 +1,76 @@
/*
* Copyright 2023 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#ifndef OBOETESTER_TEST_COLD_START_LATENCY_H
#define OBOETESTER_TEST_COLD_START_LATENCY_H
#include "oboe/Oboe.h"
#include <thread>
/**
* Test for getting the cold start latency
*/
class TestColdStartLatency {
public:
int32_t open(bool useInput, bool useLowLatency, bool useMmap, bool useExclusive);
int32_t start();
int32_t close();
int32_t getColdStartTimeMicros();
int32_t getOpenTimeMicros() {
return (int32_t) (mOpenTimeMicros.load());
}
int32_t getStartTimeMicros() {
return (int32_t) (mStartTimeMicros.load());
}
int32_t getDeviceId() {
return mDeviceId;
}
protected:
std::atomic<int64_t> mBeginStartNanos{0};
std::atomic<double> mOpenTimeMicros{0};
std::atomic<double> mStartTimeMicros{0};
std::atomic<double> mColdStartTimeMicros{0};
std::atomic<int32_t> mDeviceId{0};
private:
class MyDataCallback : public oboe::AudioStreamDataCallback { public:
MyDataCallback() {}
oboe::DataCallbackResult onAudioReady(
oboe::AudioStream *audioStream,
void *audioData,
int32_t numFrames) override;
private:
// For sine generator.
float mPhase = 0.0f;
static constexpr float kPhaseIncrement = 2.0f * (float) M_PI * 440.0f / 48000.0f;
};
std::shared_ptr<oboe::AudioStream> mStream;
std::shared_ptr<MyDataCallback> mDataCallback;
static constexpr int kChannelCount = 1;
};
#endif //OBOETESTER_TEST_COLD_START_LATENCY_H
@@ -0,0 +1,75 @@
/*
* Copyright 2022 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#include <stdlib.h>
#include "common/OboeDebug.h"
#include "TestErrorCallback.h"
using namespace oboe;
oboe::Result TestErrorCallback::open() {
mCallbackMagic = 0;
mDataCallback = std::make_shared<MyDataCallback>();
mErrorCallback = std::make_shared<MyErrorCallback>(this);
AudioStreamBuilder builder;
oboe::Result result = builder.setSharingMode(oboe::SharingMode::Exclusive)
->setPerformanceMode(oboe::PerformanceMode::LowLatency)
->setFormat(oboe::AudioFormat::Float)
->setChannelCount(kChannelCount)
#if 0
->setDataCallback(mDataCallback.get())
->setErrorCallback(mErrorCallback.get()) // This can lead to a crash or FAIL.
#else
->setDataCallback(mDataCallback)
->setErrorCallback(mErrorCallback) // shared_ptr avoids a crash
#endif
->openStream(mStream);
return result;
}
oboe::Result TestErrorCallback::start() {
return mStream->requestStart();
}
oboe::Result TestErrorCallback::stop() {
return mStream->requestStop();
}
oboe::Result TestErrorCallback::close() {
return mStream->close();
}
int TestErrorCallback::test() {
oboe::Result result = open();
if (result != oboe::Result::OK) {
return (int) result;
}
return (int) start();
}
DataCallbackResult TestErrorCallback::MyDataCallback::onAudioReady(
AudioStream *audioStream,
void *audioData,
int32_t numFrames) {
float *output = (float *) audioData;
// Fill buffer with random numbers to create "white noise".
int numSamples = numFrames * kChannelCount;
for (int i = 0; i < numSamples; i++) {
*output++ = (float)((drand48() - 0.5) * 0.2);
}
return oboe::DataCallbackResult::Continue;
}
@@ -0,0 +1,113 @@
/*
* Copyright 2022 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#ifndef OBOETESTER_TEST_ERROR_CALLBACK_H
#define OBOETESTER_TEST_ERROR_CALLBACK_H
#include "oboe/Oboe.h"
#include <thread>
/**
* This code is an experiment to see if we can cause a crash from the ErrorCallback.
*/
class TestErrorCallback {
public:
oboe::Result open();
oboe::Result start();
oboe::Result stop();
oboe::Result close();
int test();
int32_t getCallbackMagic() {
return mCallbackMagic.load();
}
protected:
std::atomic<int32_t> mCallbackMagic{0};
private:
void cleanup() {
mDataCallback.reset();
mErrorCallback.reset();
mStream.reset();
}
class MyDataCallback : public oboe::AudioStreamDataCallback { public:
oboe::DataCallbackResult onAudioReady(
oboe::AudioStream *audioStream,
void *audioData,
int32_t numFrames) override;
};
class MyErrorCallback : public oboe::AudioStreamErrorCallback {
public:
MyErrorCallback(TestErrorCallback *parent): mParent(parent) {}
virtual ~MyErrorCallback() {
// If the delete occurs before onErrorAfterClose() then this bad magic
// value will be seen by the Java test code, causing a failure.
// It is also possible that this code will just cause OboeTester to crash!
mMagic = 0xdeadbeef;
LOGE("%s() called", __func__);
}
void onErrorBeforeClose(oboe::AudioStream *oboeStream, oboe::Result error) override {
LOGE("%s() - error = %s, parent = %p",
__func__, oboe::convertToText(error), &mParent);
// Trigger a crash by "deleting" this callback object while in use!
// Do not try this at home. We are just trying to reproduce the crash
// reported in #1603.
std::thread t([this]() {
this->mParent->cleanup(); // Possibly delete stream and callback objects.
LOGE("onErrorBeforeClose called cleanup!");
});
t.detach();
// There is a race condition between the deleting thread and this thread.
// We do not want to add synchronization because the object is getting deleted
// and cannot be relied on.
// So we sleep here to give the deleting thread a chance to win the race.
usleep(10 * 1000);
}
void onErrorAfterClose(oboe::AudioStream *oboeStream, oboe::Result error) override {
// The callback was probably deleted by now.
LOGE("%s() - error = %s, mMagic = 0x%08X",
__func__, oboe::convertToText(error), mMagic.load());
mParent->mCallbackMagic = mMagic.load();
}
private:
TestErrorCallback *mParent;
// This must match the value in TestErrorCallbackActivity.java
static constexpr int32_t kMagicGood = 0x600DCAFE;
std::atomic<int32_t> mMagic{kMagicGood};
};
std::shared_ptr<oboe::AudioStream> mStream;
std::shared_ptr<MyDataCallback> mDataCallback;
std::shared_ptr<MyErrorCallback> mErrorCallback;
static constexpr int kChannelCount = 2;
};
#endif //OBOETESTER_TEST_ERROR_CALLBACK_H
@@ -0,0 +1,97 @@
/*
* Copyright 2023 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#include <stdlib.h>
#include <aaudio/AAudioExtensions.h>
#include "common/OboeDebug.h"
#include "oboe/AudioClock.h"
#include "TestRapidCycle.h"
using namespace oboe;
// start a thread to cycle through stream tests
int32_t TestRapidCycle::start(bool useOpenSL) {
mThreadEnabled = true;
mCycleCount = 0;
mCycleThread = std::thread([this, useOpenSL]() {
cycleRapidly(useOpenSL);
});
return 0;
}
int32_t TestRapidCycle::stop() {
mThreadEnabled = false;
mCycleThread.join();
return 0;
}
void TestRapidCycle::cycleRapidly(bool useOpenSL) {
while(mThreadEnabled && (oneCycle(useOpenSL) == 0));
}
int32_t TestRapidCycle::oneCycle(bool useOpenSL) {
mCycleCount++;
mDataCallback = std::make_shared<MyDataCallback>();
AudioStreamBuilder builder;
oboe::Result result = builder.setFormat(oboe::AudioFormat::Float)
->setAudioApi(useOpenSL ? oboe::AudioApi::OpenSLES : oboe::AudioApi::AAudio)
->setPerformanceMode(oboe::PerformanceMode::LowLatency)
->setChannelCount(kChannelCount)
->setDataCallback(mDataCallback)
->setUsage(oboe::Usage::Notification)
->openStream(mStream);
if (result != oboe::Result::OK) {
return (int32_t) result;
}
mStream->setDelayBeforeCloseMillis(0);
result = mStream->requestStart();
if (result != oboe::Result::OK) {
mStream->close();
return (int32_t) result;
}
// Sleep for some random time.
int32_t durationMicros = (int32_t)(drand48() * kMaxSleepMicros);
LOGD("TestRapidCycle::oneCycle() - Sleep for %d micros", durationMicros);
usleep(durationMicros);
LOGD("TestRapidCycle::oneCycle() - Woke up, close stream");
mDataCallback->returnStop = true;
result = mStream->close();
return (int32_t) result;
}
// Callback that sleeps then touches the audio buffer.
DataCallbackResult TestRapidCycle::MyDataCallback::onAudioReady(
AudioStream *audioStream,
void *audioData,
int32_t numFrames) {
float *floatData = (float *) audioData;
const int numSamples = numFrames * kChannelCount;
// Fill buffer with white noise.
for (int i = 0; i < numSamples; i++) {
floatData[i] = ((float) drand48() - 0.5f) * 2 * 0.1f;
}
usleep(500); // half a millisecond
if (returnStop) {
usleep(20 * 1000);
return DataCallbackResult::Stop;
} else {
return DataCallbackResult::Continue;
}
}
@@ -0,0 +1,67 @@
/*
* Copyright 2023 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#ifndef OBOETESTER_TEST_RAPID_CYCLE_H
#define OBOETESTER_TEST_RAPID_CYCLE_H
#include "oboe/Oboe.h"
#include <thread>
/**
* Try to cause a crash by changing routing during a data callback.
* We use Use::VoiceCommunication for the stream and
* setSpeakerPhoneOn(b) to force a routing change.
* This works best when connected to a BT headset.
*/
class TestRapidCycle {
public:
int32_t start(bool useOpenSL);
int32_t stop();
int32_t getCycleCount() {
return mCycleCount.load();
}
private:
void cycleRapidly(bool useOpenSL);
int32_t oneCycle(bool useOpenSL);
class MyDataCallback : public oboe::AudioStreamDataCallback { public:
MyDataCallback() {}
oboe::DataCallbackResult onAudioReady(
oboe::AudioStream *audioStream,
void *audioData,
int32_t numFrames) override;
bool returnStop = false;
};
std::shared_ptr<oboe::AudioStream> mStream;
std::shared_ptr<MyDataCallback> mDataCallback;
std::atomic<int32_t> mCycleCount{0};
std::atomic<bool> mThreadEnabled{false};
std::thread mCycleThread;
static constexpr int kChannelCount = 1;
static constexpr int kMaxSleepMicros = 25000;
};
#endif //OBOETESTER_TEST_RAPID_CYCLE_H
@@ -0,0 +1,111 @@
/*
* Copyright 2023 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#include <stdlib.h>
#include <aaudio/AAudioExtensions.h>
#include "common/OboeDebug.h"
#include "oboe/AudioClock.h"
#include "TestRoutingCrash.h"
using namespace oboe;
// open start start an Oboe stream
int32_t TestRoutingCrash::start(bool useInput) {
mDataCallback = std::make_shared<MyDataCallback>(this);
// Disable MMAP because we are trying to crash a Legacy Stream.
bool wasMMapEnabled = AAudioExtensions::getInstance().isMMapEnabled();
AAudioExtensions::getInstance().setMMapEnabled(false);
AudioStreamBuilder builder;
oboe::Result result = builder.setFormat(oboe::AudioFormat::Float)
#if 1
->setPerformanceMode(oboe::PerformanceMode::LowLatency)
#else
->setPerformanceMode(oboe::PerformanceMode::None)
#endif
->setDirection(useInput ? oboe::Direction::Input : oboe::Direction::Output)
->setChannelCount(kChannelCount)
->setDataCallback(mDataCallback)
// Use VoiceCommunication so we can reroute it by setting SpeakerPhone ON/OFF.
->setUsage(oboe::Usage::VoiceCommunication)
->openStream(mStream);
if (result != oboe::Result::OK) {
return (int32_t) result;
}
AAudioExtensions::getInstance().setMMapEnabled(wasMMapEnabled);
return (int32_t) mStream->requestStart();
}
int32_t TestRoutingCrash::stop() {
oboe::Result result1 = mStream->requestStop();
oboe::Result result2 = mStream->close();
return (int32_t)((result1 != oboe::Result::OK) ? result1 : result2);
}
// Callback that sleeps then touches the audio buffer.
DataCallbackResult TestRoutingCrash::MyDataCallback::onAudioReady(
AudioStream *audioStream,
void *audioData,
int32_t numFrames) {
float *floatData = (float *) audioData;
// If I call getTimestamp() here it does NOT crash!
// Simulate the timing of a heavy workload by sleeping.
// Otherwise the window for the crash is very narrow.
const double kDutyCycle = 0.7;
const double bufferTimeNanos = 1.0e9 * numFrames / (double) audioStream->getSampleRate();
const int64_t targetDurationNanos = (int64_t) (bufferTimeNanos * kDutyCycle);
if (targetDurationNanos > 0) {
AudioClock::sleepForNanos(targetDurationNanos);
}
const double kFilterCoefficient = 0.95; // low pass IIR filter
const double sleepMicros = targetDurationNanos * 0.0001;
mParent->averageSleepTimeMicros = ((1.0 - kFilterCoefficient) * sleepMicros)
+ (kFilterCoefficient * mParent->averageSleepTimeMicros);
// If I call getTimestamp() here it crashes.
audioStream->getTimestamp(CLOCK_MONOTONIC); // Trigger a restoreTrack_l() in framework.
const int numSamples = numFrames * kChannelCount;
if (audioStream->getDirection() == oboe::Direction::Input) {
// Read buffer and write sum of samples to a member variable.
// We just want to touch the memory and not get optimized away by the compiler.
float sum = 0.0f;
for (int i = 0; i < numSamples; i++) {
sum += *floatData++;
}
mInputSum = sum;
} else {
// Fill mono buffer with a sine wave.
// If the routing occurred then the buffer may be dead and
// we may be writing into unallocated memory.
for (int i = 0; i < numSamples; i++) {
*floatData++ = sinf(mPhase) * 0.2f;
mPhase += kPhaseIncrement;
// Wrap the phase around in a circle.
if (mPhase >= M_PI) mPhase -= 2 * M_PI;
}
}
// If I call getTimestamp() here it does NOT crash!
return oboe::DataCallbackResult::Continue;
}
@@ -0,0 +1,67 @@
/*
* Copyright 2023 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#ifndef OBOETESTER_TEST_ROUTING_CRASH_H
#define OBOETESTER_TEST_ROUTING_CRASH_H
#include "oboe/Oboe.h"
#include <thread>
/**
* Try to cause a crash by changing routing during a data callback.
* We use Use::VoiceCommunication for the stream and
* setSpeakerPhoneOn(b) to force a routing change.
* This works best when connected to a BT headset.
*/
class TestRoutingCrash {
public:
int32_t start(bool useInput);
int32_t stop();
int32_t getSleepTimeMicros() {
return (int32_t) (averageSleepTimeMicros.load());
}
protected:
std::atomic<double> averageSleepTimeMicros{0};
private:
class MyDataCallback : public oboe::AudioStreamDataCallback { public:
MyDataCallback(TestRoutingCrash *parent): mParent(parent) {}
oboe::DataCallbackResult onAudioReady(
oboe::AudioStream *audioStream,
void *audioData,
int32_t numFrames) override;
private:
TestRoutingCrash *mParent;
// For sine generator.
float mPhase = 0.0f;
static constexpr float kPhaseIncrement = 2.0f * (float) M_PI * 440.0f / 48000.0f;
float mInputSum = 0.0f; // For saving input data sum to prevent over-optimization.
};
std::shared_ptr<oboe::AudioStream> mStream;
std::shared_ptr<MyDataCallback> mDataCallback;
static constexpr int kChannelCount = 1;
};
#endif //OBOETESTER_TEST_ROUTING_CRASH_H
@@ -0,0 +1,240 @@
/*
* Copyright (C) 2020 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#ifndef ANALYZER_BASE_SINE_ANALYZER_H
#define ANALYZER_BASE_SINE_ANALYZER_H
#include <algorithm>
#include <cctype>
#include <iomanip>
#include <iostream>
#include "InfiniteRecording.h"
#include "LatencyAnalyzer.h"
/**
* Output a steady sine wave and analyze the return signal.
*
* Use a cosine transform to measure the predicted magnitude and relative phase of the
* looped back sine wave. Then generate a predicted signal and compare with the actual signal.
*/
class BaseSineAnalyzer : public LoopbackProcessor {
public:
BaseSineAnalyzer()
: LoopbackProcessor()
, mInfiniteRecording(64 * 1024) {}
virtual bool isOutputEnabled() { return true; }
void setMagnitude(double magnitude) {
mMagnitude = magnitude;
mScaledTolerance = mMagnitude * getTolerance();
}
/**
*
* @return valid phase or kPhaseInvalid=-999
*/
double getPhaseOffset() {
ALOGD("%s(), mPhaseOffset = %f\n", __func__, mPhaseOffset);
return mPhaseOffset;
}
double getMagnitude() const {
return mMagnitude;
}
void setNoiseAmplitude(double noiseAmplitude) {
mNoiseAmplitude = noiseAmplitude;
}
double getNoiseAmplitude() const {
return mNoiseAmplitude;
}
double getTolerance() {
return mTolerance;
}
void setTolerance(double tolerance) {
mTolerance = tolerance;
}
// advance and wrap phase
void incrementInputPhase() {
mInputPhase += mPhaseIncrement;
if (mInputPhase > M_PI) {
mInputPhase -= (2.0 * M_PI);
}
}
// advance and wrap phase
void incrementOutputPhase() {
mOutputPhase += mPhaseIncrement;
if (mOutputPhase > M_PI) {
mOutputPhase -= (2.0 * M_PI);
}
}
/**
* @param frameData upon return, contains the reference sine wave
* @param channelCount
*/
result_code processOutputFrame(float *frameData, int channelCount) override {
float output = 0.0f;
// Output sine wave so we can measure it.
if (isOutputEnabled()) {
float sinOut = sinf(mOutputPhase);
incrementOutputPhase();
output = (sinOut * mOutputAmplitude)
+ (mWhiteNoise.nextRandomDouble() * getNoiseAmplitude());
// ALOGD("sin(%f) = %f, %f\n", mOutputPhase, sinOut, kPhaseIncrement);
}
for (int i = 0; i < channelCount; i++) {
frameData[i] = (i == getOutputChannel()) ? output : 0.0f;
}
return RESULT_OK;
}
/**
* Calculate the magnitude of the component of the input signal
* that matches the analysis frequency.
* Also calculate the phase that we can use to create a
* signal that matches that component.
* The phase will be between -PI and +PI.
*/
double calculateMagnitudePhase(double *phasePtr = nullptr) {
if (mFramesAccumulated == 0) {
return 0.0;
}
double sinMean = mSinAccumulator / mFramesAccumulated;
double cosMean = mCosAccumulator / mFramesAccumulated;
double magnitude = 2.0 * sqrt((sinMean * sinMean) + (cosMean * cosMean));
if (phasePtr != nullptr) {
double phase;
if (magnitude < kMinValidMagnitude) {
phase = kPhaseInvalid;
ALOGD("%s() mag very low! sinMean = %7.5f, cosMean = %7.5f",
__func__, sinMean, cosMean);
} else {
phase = atan2(cosMean, sinMean);
if (phase == 0.0) {
ALOGD("%s() phase zero! sinMean = %7.5f, cosMean = %7.5f",
__func__, sinMean, cosMean);
}
}
*phasePtr = phase;
}
return magnitude;
}
/**
* Perform sin/cos analysis on each sample.
* Measure magnitude and phase on every period.
* Updates mPhaseOffset
* @param sample
* @param referencePhase
* @return true if magnitude and phase updated
*/
bool transformSample(float sample) {
// Compare incoming signal with the reference input sine wave.
mSinAccumulator += static_cast<double>(sample) * sinf(mInputPhase);
mCosAccumulator += static_cast<double>(sample) * cosf(mInputPhase);
incrementInputPhase();
mFramesAccumulated++;
// Must be a multiple of the period or the calculation will not be accurate.
if (mFramesAccumulated == mSinePeriod) {
const double coefficient = 0.1;
double magnitude = calculateMagnitudePhase(&mPhaseOffset);
ALOGD("%s(), phaseOffset = %f\n", __func__, mPhaseOffset);
if (mPhaseOffset != kPhaseInvalid) {
// One pole averaging filter.
setMagnitude((mMagnitude * (1.0 - coefficient)) + (magnitude * coefficient));
}
resetAccumulator();
return true;
} else {
return false;
}
}
// reset the sine wave detector
virtual void resetAccumulator() {
mFramesAccumulated = 0;
mSinAccumulator = 0.0;
mCosAccumulator = 0.0;
}
void reset() override {
LoopbackProcessor::reset();
resetAccumulator();
mMagnitude = 0.0;
}
void prepareToTest() override {
LoopbackProcessor::prepareToTest();
mSinePeriod = getSampleRate() / kTargetGlitchFrequency;
mInputPhase = 0.0f;
mOutputPhase = 0.0f;
mInverseSinePeriod = 1.0 / mSinePeriod;
mPhaseIncrement = 2.0 * M_PI * mInverseSinePeriod;
}
protected:
// Try to get a prime period so the waveform plot changes every time.
static constexpr int32_t kTargetGlitchFrequency = 48000 / 113;
int32_t mSinePeriod = 1; // this will be set before use
double mInverseSinePeriod = 1.0;
double mPhaseIncrement = 0.0;
// Use two sine wave phases, input and output.
// This is because the number of input and output samples may differ
// in a callback and the output frame count may advance ahead of the input, or visa versa.
double mInputPhase = 0.0;
double mOutputPhase = 0.0;
double mOutputAmplitude = 0.75;
// This is the phase offset between the mInputPhase sine wave and the recorded
// signal at the tuned frequency.
// If this jumps around then we are probably just hearing noise.
// Noise can cause the magnitude to be high but mPhaseOffset will be pretty random.
// If we are tracking a sine wave then mPhaseOffset should be consistent.
double mPhaseOffset = 0.0;
// kPhaseInvalid indicates that the phase measurement cannot be used.
// We were seeing times when a magnitude of zero was causing atan2(s,c) to
// return a phase of zero, which looked valid to Java. This is a way of passing
// an error code back to Java as a single value to avoid race conditions.
static constexpr double kPhaseInvalid = -999.0;
double mMagnitude = 0.0;
static constexpr double kMinValidMagnitude = 2.0 / (1 << 16);
int32_t mFramesAccumulated = 0;
double mSinAccumulator = 0.0;
double mCosAccumulator = 0.0;
double mScaledTolerance = 0.0;
InfiniteRecording<float> mInfiniteRecording;
private:
float mTolerance = 0.10; // scaled from 0.0 to 1.0
float mNoiseAmplitude = 0.00; // Used to experiment with warbling caused by DRC.
PseudoRandom mWhiteNoise;
};
#endif //ANALYZER_BASE_SINE_ANALYZER_H
@@ -0,0 +1,106 @@
/*
* Copyright (C) 2020 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#ifndef ANALYZER_DATA_PATH_ANALYZER_H
#define ANALYZER_DATA_PATH_ANALYZER_H
#include <algorithm>
#include <cctype>
#include <iomanip>
#include <iostream>
#include <math.h>
#include "BaseSineAnalyzer.h"
#include "InfiniteRecording.h"
#include "LatencyAnalyzer.h"
/**
* Output a steady sine wave and analyze the return signal.
*
* Use a cosine transform to measure the predicted magnitude and relative phase of the
* looped back sine wave.
*/
class DataPathAnalyzer : public BaseSineAnalyzer {
public:
DataPathAnalyzer() : BaseSineAnalyzer() {
// Add a little bit of noise to reduce blockage by speaker protection and DRC.
setNoiseAmplitude(0.02);
}
double calculatePhaseError(double p1, double p2) {
double diff = p1 - p2;
// Wrap around the circle.
while (diff > M_PI) {
diff -= (2 * M_PI);
}
while (diff < -M_PI) {
diff += (2 * M_PI);
}
return diff;
}
/**
* @param frameData contains microphone data with sine signal feedback
* @param channelCount
*/
result_code processInputFrame(const float *frameData, int /* channelCount */) override {
result_code result = RESULT_OK;
float sample = frameData[getInputChannel()];
mInfiniteRecording.write(sample);
if (transformSample(sample)) {
// Analyze magnitude and phase on every period.
if (mPhaseOffset != kPhaseInvalid) {
double diff = fabs(calculatePhaseError(mPhaseOffset, mPreviousPhaseOffset));
if (diff < mPhaseTolerance) {
mMaxMagnitude = std::max(mMagnitude, mMaxMagnitude);
}
mPreviousPhaseOffset = mPhaseOffset;
}
}
return result;
}
std::string analyze() override {
std::stringstream report;
report << "DataPathAnalyzer ------------------\n";
report << LOOPBACK_RESULT_TAG "sine.magnitude = " << std::setw(8)
<< mMagnitude << "\n";
report << LOOPBACK_RESULT_TAG "frames.accumulated = " << std::setw(8)
<< mFramesAccumulated << "\n";
report << LOOPBACK_RESULT_TAG "sine.period = " << std::setw(8)
<< mSinePeriod << "\n";
return report.str();
}
void reset() override {
BaseSineAnalyzer::reset();
mPreviousPhaseOffset = 999.0; // Arbitrary high offset to prevent early lock.
mMaxMagnitude = 0.0;
}
double getMaxMagnitude() {
return mMaxMagnitude;
}
private:
double mPreviousPhaseOffset = 0.0;
double mPhaseTolerance = 2 * M_PI / 48;
double mMaxMagnitude = 0.0;
};
#endif // ANALYZER_DATA_PATH_ANALYZER_H
@@ -14,141 +14,140 @@
* limitations under the License.
*/
#ifndef OBOETESTER_GLITCHANALYZER_H
#define OBOETESTER_GLITCHANALYZER_H
#ifndef ANALYZER_GLITCH_ANALYZER_H
#define ANALYZER_GLITCH_ANALYZER_H
#include <algorithm>
#include <cctype>
#include <iomanip>
#include <iostream>
#include "PseudoRandom.h"
#include "LatencyAnalyzer.h"
#include "InfiniteRecording.h"
#include "LatencyAnalyzer.h"
#include "BaseSineAnalyzer.h"
#include "PseudoRandom.h"
/**
* Output a steady sinewave and analyze the return signal.
* Output a steady sine wave and analyze the return signal.
*
* Use a cosine transform to measure the predicted magnitude and relative phase of the
* looped back sine wave. Then generate a predicted signal and compare with the actual signal.
*/
class GlitchAnalyzer : public LoopbackProcessor {
class GlitchAnalyzer : public BaseSineAnalyzer {
public:
GlitchAnalyzer()
: LoopbackProcessor()
, mInfiniteRecording(64 * 1024) {}
GlitchAnalyzer() : BaseSineAnalyzer() {}
int32_t getState() {
int32_t getState() const {
return mState;
}
float getPeakAmplitude() {
double getPeakAmplitude() const {
return mPeakFollower.getLevel();
}
float getTolerance() {
return mTolerance;
double getSineAmplitude() const {
return mMagnitude;
}
void setTolerance(float tolerance) {
mTolerance = tolerance;
mScaledTolerance = mMagnitude * mTolerance;
int getSinePeriod() const {
return mSinePeriod;
}
void setMagnitude(double magnitude) {
mMagnitude = magnitude;
mScaledTolerance = mMagnitude * mTolerance;
}
int32_t getGlitchCount() {
int32_t getGlitchCount() const {
return mGlitchCount;
}
int32_t getStateFrameCount(int state) {
int32_t getGlitchLength() const {
return mGlitchLength;
}
int32_t getStateFrameCount(int state) const {
return mStateFrameCounters[state];
}
double getSignalToNoiseDB() {
static const double threshold = 1.0e-14;
if (mMeanSquareSignal < threshold || mMeanSquareNoise < threshold) {
return 0.0;
if (mState != STATE_LOCKED
|| mMeanSquareSignal < threshold
|| mMeanSquareNoise < threshold) {
return -999.0; // error indicator
} else {
double signalToNoise = mMeanSquareSignal / mMeanSquareNoise; // power ratio
double signalToNoiseDB = 10.0 * log(signalToNoise);
if (signalToNoiseDB < MIN_SNRATIO_DB) {
LOGD("ERROR - signal to noise ratio is too low! < %d dB. Adjust volume.",
MIN_SNRATIO_DB);
if (signalToNoiseDB < static_cast<float>(MIN_SNR_DB)) {
setResult(ERROR_VOLUME_TOO_LOW);
}
return signalToNoiseDB;
}
}
void analyze() override {
LOGD("GlitchAnalyzer ------------------");
LOGD(LOOPBACK_RESULT_TAG "peak.amplitude = %8f", getPeakAmplitude());
LOGD(LOOPBACK_RESULT_TAG "sine.magnitude = %8f", mMagnitude);
LOGD(LOOPBACK_RESULT_TAG "rms.noise = %8f", mMeanSquareNoise);
LOGD(LOOPBACK_RESULT_TAG "signal.to.noise.db = %8.2f", getSignalToNoiseDB());
LOGD(LOOPBACK_RESULT_TAG "frames.accumulated = %8d", mFramesAccumulated);
LOGD(LOOPBACK_RESULT_TAG "sine.period = %8d", mSinePeriod);
LOGD(LOOPBACK_RESULT_TAG "test.state = %8d", mState);
LOGD(LOOPBACK_RESULT_TAG "frame.count = %8d", mFrameCounter);
std::string analyze() override {
std::stringstream report;
report << "GlitchAnalyzer ------------------\n";
report << LOOPBACK_RESULT_TAG "peak.amplitude = " << std::setw(8)
<< getPeakAmplitude() << "\n";
report << LOOPBACK_RESULT_TAG "sine.magnitude = " << std::setw(8)
<< getSineAmplitude() << "\n";
report << LOOPBACK_RESULT_TAG "rms.noise = " << std::setw(8)
<< mMeanSquareNoise << "\n";
report << LOOPBACK_RESULT_TAG "signal.to.noise.db = " << std::setw(8)
<< getSignalToNoiseDB() << "\n";
report << LOOPBACK_RESULT_TAG "frames.accumulated = " << std::setw(8)
<< mFramesAccumulated << "\n";
report << LOOPBACK_RESULT_TAG "sine.period = " << std::setw(8)
<< mSinePeriod << "\n";
report << LOOPBACK_RESULT_TAG "test.state = " << std::setw(8)
<< mState << "\n";
report << LOOPBACK_RESULT_TAG "frame.count = " << std::setw(8)
<< mFrameCounter << "\n";
// Did we ever get a lock?
bool gotLock = (mState == STATE_LOCKED) || (mGlitchCount > 0);
if (!gotLock) {
LOGD("ERROR - failed to lock on reference sine tone");
report << "ERROR - failed to lock on reference sine tone.\n";
setResult(ERROR_NO_LOCK);
} else {
// Only print if meaningful.
LOGD(LOOPBACK_RESULT_TAG "glitch.count = %8d", mGlitchCount);
LOGD(LOOPBACK_RESULT_TAG "max.glitch = %8f", mMaxGlitchDelta);
report << LOOPBACK_RESULT_TAG "glitch.count = " << std::setw(8)
<< mGlitchCount << "\n";
report << LOOPBACK_RESULT_TAG "max.glitch = " << std::setw(8)
<< mMaxGlitchDelta << "\n";
if (mGlitchCount > 0) {
LOGD("ERROR - number of glitches > 0");
report << "ERROR - number of glitches > 0\n";
setResult(ERROR_GLITCHES);
}
}
return report.str();
}
void printStatus() override {
LOGD("st = %d, #gl = %3d,", mState, mGlitchCount);
}
double calculateMagnitude(double *phasePtr = NULL) {
if (mFramesAccumulated == 0) {
return 0.0;
}
double sinMean = mSinAccumulator / mFramesAccumulated;
double cosMean = mCosAccumulator / mFramesAccumulated;
double magnitude = 2.0 * sqrt( (sinMean * sinMean) + (cosMean * cosMean ));
if( phasePtr != NULL )
{
double phase = M_PI_2 - atan2( sinMean, cosMean );
*phasePtr = phase;
}
return magnitude;
ALOGD("st = %d, #gl = %3d,", mState, mGlitchCount);
}
/**
* @param frameData contains microphone data with sine signal feedback
* @param channelCount
*/
result_code processInputFrame(float *frameData, int channelCount) override {
result_code processInputFrame(const float *frameData, int /* channelCount */) override {
result_code result = RESULT_OK;
float sample = frameData[0];
float peak = mPeakFollower.process(sample);
mInfiniteRecording.write(sample);
float sample = frameData[getInputChannel()];
// Force a periodic glitch!
if (mForceGlitchDuration > 0) {
// Force a periodic glitch to test the detector!
if (mForceGlitchDurationFrames > 0) {
if (mForceGlitchCounter == 0) {
LOGE("%s: force a glitch!!", __func__);
mForceGlitchCounter = getSampleRate();
} else if (mForceGlitchCounter <= mForceGlitchDuration) {
sample += (sample > 0.0) ? -0.5f : 0.5f;
ALOGE("%s: finish a glitch!!", __func__);
mForceGlitchCounter = kForceGlitchPeriod;
} else if (mForceGlitchCounter <= mForceGlitchDurationFrames) {
// Force an abrupt offset.
sample += (sample > 0.0) ? -kForceGlitchOffset : kForceGlitchOffset;
}
--mForceGlitchCounter;
}
float peak = mPeakFollower.process(sample);
mInfiniteRecording.write(sample);
mStateFrameCounters[mState]++; // count how many frames we are in each state
switch (mState) {
@@ -159,6 +158,7 @@ public:
mDownCounter = IMMUNE_FRAME_COUNT;
mInputPhase = 0.0; // prevent spike at start
mOutputPhase = 0.0;
resetAccumulator();
}
break;
@@ -172,28 +172,31 @@ public:
case STATE_WAITING_FOR_SIGNAL:
if (peak > mThreshold) {
mState = STATE_WAITING_FOR_LOCK;
//LOGD("%5d: switch to STATE_WAITING_FOR_LOCK", mFrameCounter);
//ALOGD("%5d: switch to STATE_WAITING_FOR_LOCK", mFrameCounter);
resetAccumulator();
}
break;
case STATE_WAITING_FOR_LOCK:
mSinAccumulator += sample * sinf(mInputPhase);
mCosAccumulator += sample * cosf(mInputPhase);
mSinAccumulator += static_cast<double>(sample) * sinf(mInputPhase);
mCosAccumulator += static_cast<double>(sample) * cosf(mInputPhase);
mFramesAccumulated++;
// Must be a multiple of the period or the calculation will not be accurate.
if (mFramesAccumulated == mSinePeriod * PERIODS_NEEDED_FOR_LOCK) {
double phaseOffset = 0.0;
setMagnitude(calculateMagnitude(&phaseOffset));
// LOGD("%s() mag = %f, offset = %f, prev = %f",
// __func__, mMagnitude, mPhaseOffset, mPreviousPhaseOffset);
if (mMagnitude > mThreshold) {
if (abs(phaseOffset) < kMaxPhaseError) {
mState = STATE_LOCKED;
// LOGD("%5d: switch to STATE_LOCKED", mFrameCounter);
double magnitude = calculateMagnitudePhase(&mPhaseOffset);
if (mPhaseOffset != kPhaseInvalid) {
setMagnitude(magnitude);
ALOGD("%s() mag = %f, mPhaseOffset = %f",
__func__, magnitude, mPhaseOffset);
if (mMagnitude > mThreshold) {
if (fabs(mPhaseOffset) < kMaxPhaseError) {
mState = STATE_LOCKED;
mConsecutiveBadFrames = 0;
// ALOGD("%5d: switch to STATE_LOCKED", mFrameCounter);
}
// Adjust mInputPhase to match measured phase
mInputPhase += mPhaseOffset;
}
// Adjust mInputPhase to match measured phase
mInputPhase += phaseOffset;
}
resetAccumulator();
}
@@ -202,65 +205,72 @@ public:
case STATE_LOCKED: {
// Predict next sine value
float predicted = sinf(mInputPhase) * mMagnitude;
float diff = predicted - sample;
float absDiff = fabs(diff);
double predicted = sinf(mInputPhase) * mMagnitude;
double diff = predicted - sample;
double absDiff = fabs(diff);
mMaxGlitchDelta = std::max(mMaxGlitchDelta, absDiff);
if (absDiff > mScaledTolerance) {
result = ERROR_GLITCHES;
onGlitchStart();
// LOGI("diff glitch detected, absDiff = %g", absDiff);
} else {
if (absDiff > mScaledTolerance) { // bad frame
mConsecutiveBadFrames++;
mConsecutiveGoodFrames = 0;
LOGI("diff glitch frame #%d detected, absDiff = %g > %g",
mConsecutiveBadFrames, absDiff, mScaledTolerance);
if (mConsecutiveBadFrames > 0) {
result = ERROR_GLITCHES;
onGlitchStart();
}
resetAccumulator();
} else { // good frame
mConsecutiveBadFrames = 0;
mConsecutiveGoodFrames++;
mSumSquareSignal += predicted * predicted;
mSumSquareNoise += diff * diff;
// Track incoming signal and slowly adjust magnitude to account
// for drift in the DRC or AGC.
mSinAccumulator += sample * sinf(mInputPhase);
mCosAccumulator += sample * cosf(mInputPhase);
mFramesAccumulated++;
// Must be a multiple of the period or the calculation will not be accurate.
if (mFramesAccumulated == mSinePeriod) {
const double coefficient = 0.1;
double phaseOffset = 0.0;
double magnitude = calculateMagnitude(&phaseOffset);
// One pole averaging filter.
setMagnitude((mMagnitude * (1.0 - coefficient)) + (magnitude * coefficient));
if (transformSample(sample)) {
// Adjust phase to account for sample rate drift.
mInputPhase += mPhaseOffset;
mMeanSquareNoise = mSumSquareNoise * mInverseSinePeriod;
mMeanSquareSignal = mSumSquareSignal * mInverseSinePeriod;
resetAccumulator();
mSumSquareNoise = 0.0;
mSumSquareSignal = 0.0;
if (abs(phaseOffset) > kMaxPhaseError) {
if (fabs(mPhaseOffset) > kMaxPhaseError) {
result = ERROR_GLITCHES;
onGlitchStart();
LOGD("phase glitch detected, phaseOffset = %g", phaseOffset);
ALOGD("phase glitch detected, phaseOffset = %g", mPhaseOffset);
} else if (mMagnitude < mThreshold) {
result = ERROR_GLITCHES;
onGlitchStart();
LOGD("magnitude glitch detected, mMagnitude = %g", mMagnitude);
ALOGD("magnitude glitch detected, mMagnitude = %g", mMagnitude);
}
}
}
incrementInputPhase();
} break;
case STATE_GLITCHING: {
// Predict next sine value
mGlitchLength++;
float predicted = sinf(mInputPhase) * mMagnitude;
float diff = predicted - sample;
float absDiff = fabs(diff);
double predicted = sinf(mInputPhase) * mMagnitude;
double diff = predicted - sample;
double absDiff = fabs(diff);
mMaxGlitchDelta = std::max(mMaxGlitchDelta, absDiff);
if (absDiff < mScaledTolerance) { // close enough?
// If we get a full sine period of non-glitch samples in a row then consider the glitch over.
// We don't want to just consider a zero crossing the end of a glitch.
if (mNonGlitchCount++ > mSinePeriod) {
onGlitchEnd();
if (absDiff > mScaledTolerance) { // bad frame
mConsecutiveBadFrames++;
mConsecutiveGoodFrames = 0;
mGlitchLength++;
if (mGlitchLength > maxMeasurableGlitchLength()) {
onGlitchTerminated();
}
} else {
mNonGlitchCount = 0;
if (mGlitchLength > (4 * mSinePeriod)) {
relock();
} else { // good frame
mConsecutiveBadFrames = 0;
mConsecutiveGoodFrames++;
// If we get a full sine period of good samples in a row then consider the glitch over.
// We don't want to just consider a zero crossing the end of a glitch.
if (mConsecutiveGoodFrames > mSinePeriod) {
onGlitchEnd();
}
}
incrementInputPhase();
@@ -275,96 +285,80 @@ public:
return result;
}
// advance and wrap phase
void incrementInputPhase() {
mInputPhase += mPhaseIncrement;
if (mInputPhase > M_PI) {
mInputPhase -= (2.0 * M_PI);
}
}
int maxMeasurableGlitchLength() const { return 2 * mSinePeriod; }
// advance and wrap phase
void incrementOutputPhase() {
mOutputPhase += mPhaseIncrement;
if (mOutputPhase > M_PI) {
mOutputPhase -= (2.0 * M_PI);
}
bool isOutputEnabled() override { return mState != STATE_IDLE; }
void onGlitchStart() {
mState = STATE_GLITCHING;
mGlitchLength = 1;
mLastGlitchPosition = mInfiniteRecording.getTotalWritten();
ALOGD("%5d: STARTED a glitch # %d, pos = %5d",
mFrameCounter, mGlitchCount, (int)mLastGlitchPosition);
ALOGD("glitch mSinePeriod = %d", mSinePeriod);
}
/**
* @param frameData upon return, contains the reference sine wave
* @param channelCount
* Give up waiting for a glitch to end and try to resync.
*/
result_code processOutputFrame(float *frameData, int channelCount) override {
float output = 0.0f;
// Output sine wave so we can measure it.
if (mState != STATE_IDLE) {
float sinOut = sinf(mOutputPhase);
incrementOutputPhase();
output = (sinOut * mOutputAmplitude)
+ (mWhiteNoise.nextRandomDouble() * kNoiseAmplitude);
// LOGD("%5d: sin(%f) = %f, %f", i, mPhase, sinOut, mPhaseIncrement);
}
frameData[0] = output;
for (int i = 1; i < channelCount; i++) {
frameData[i] = 0.0f;
}
return RESULT_OK;
}
void onGlitchStart() {
void onGlitchTerminated() {
mGlitchCount++;
// LOGD("%5d: STARTED a glitch # %d", mFrameCounter, mGlitchCount);
mState = STATE_GLITCHING;
mGlitchLength = 1;
mNonGlitchCount = 0;
mLastGlitchPosition = mInfiniteRecording.getTotalWritten();
ALOGD("%5d: TERMINATED a glitch # %d, length = %d", mFrameCounter, mGlitchCount, mGlitchLength);
// We don't know how long the glitch really is so set the length to -1.
mGlitchLength = -1;
mState = STATE_WAITING_FOR_LOCK;
resetAccumulator();
}
void onGlitchEnd() {
// LOGD("%5d: ENDED a glitch # %d, length = %d", mFrameCounter, mGlitchCount, mGlitchLength);
mGlitchCount++;
ALOGD("%5d: ENDED a glitch # %d, length = %d", mFrameCounter, mGlitchCount, mGlitchLength);
mState = STATE_LOCKED;
resetAccumulator();
}
// reset the sine wave detector
void resetAccumulator() {
mFramesAccumulated = 0;
mSinAccumulator = 0.0;
mCosAccumulator = 0.0;
mSumSquareSignal = 0.0;
mSumSquareNoise = 0.0;
}
void relock() {
// LOGD("relock: %d because of a very long %d glitch", mFrameCounter, mGlitchLength);
mState = STATE_WAITING_FOR_LOCK;
resetAccumulator();
void resetAccumulator() override {
BaseSineAnalyzer::resetAccumulator();
}
void reset() override {
LoopbackProcessor::reset();
BaseSineAnalyzer::reset();
mState = STATE_IDLE;
mDownCounter = IDLE_FRAME_COUNT;
resetAccumulator();
}
void onStartTest() override {
LoopbackProcessor::onStartTest();
mSinePeriod = getSampleRate() / kTargetGlitchFrequency;
mOutputPhase = 0.0f;
mInverseSinePeriod = 1.0 / mSinePeriod;
mPhaseIncrement = 2.0 * M_PI * mInverseSinePeriod;
void prepareToTest() override {
BaseSineAnalyzer::prepareToTest();
mGlitchCount = 0;
mGlitchLength = 0;
mMaxGlitchDelta = 0.0;
for (int i = 0; i < NUM_STATES; i++) {
mStateFrameCounters[i] = 0;
}
}
int32_t getLastGlitch(float *buffer, int32_t length) {
return mInfiniteRecording.readFrom(buffer, mLastGlitchPosition - 32, length);
const int margin = mSinePeriod;
int32_t numSamples = mInfiniteRecording.readFrom(buffer,
mLastGlitchPosition - margin,
length);
ALOGD("%s: glitch at %d, edge = %7.4f, %7.4f, %7.4f",
__func__, (int)mLastGlitchPosition,
buffer[margin - 1], buffer[margin], buffer[margin+1]);
return numSamples;
}
int32_t getRecentSamples(float *buffer, int32_t length) {
int firstSample = mInfiniteRecording.getTotalWritten() - length;
int32_t numSamples = mInfiniteRecording.readFrom(buffer,
firstSample,
length);
return numSamples;
}
void setForcedGlitchDuration(int frames) {
mForceGlitchDurationFrames = frames;
}
private:
@@ -372,51 +366,42 @@ private:
// These must match the values in GlitchActivity.java
enum sine_state_t {
STATE_IDLE, // beginning
STATE_IMMUNE, // ignoring input, waiting fo HW to settle
STATE_IMMUNE, // ignoring input, waiting for HW to settle
STATE_WAITING_FOR_SIGNAL, // looking for a loud signal
STATE_WAITING_FOR_LOCK, // trying to lock onto the phase of the sine
STATE_LOCKED, // locked on the sine wave, looking for glitches
STATE_GLITCHING, // locked on the sine wave but glitching
STATE_GLITCHING, // locked on the sine wave but glitching
NUM_STATES
};
enum constants {
// Arbitrary durations, assuming 48000 Hz
IDLE_FRAME_COUNT = 48 * 100,
IDLE_FRAME_COUNT = 48 * 100,
IMMUNE_FRAME_COUNT = 48 * 100,
PERIODS_NEEDED_FOR_LOCK = 8,
MIN_SNRATIO_DB = 65
MIN_SNR_DB = 65
};
static constexpr float kNoiseAmplitude = 0.00; // Used to experiment with warbling caused by DRC.
static constexpr int kTargetGlitchFrequency = 607;
static constexpr double kMaxPhaseError = M_PI * 0.05;
float mTolerance = 0.10; // scaled from 0.0 to 1.0
double mThreshold = 0.005;
int mSinePeriod = 1; // this will be set before use
double mInverseSinePeriod = 1.0;
int32_t mStateFrameCounters[NUM_STATES];
sine_state_t mState = STATE_IDLE;
int64_t mLastGlitchPosition;
double mPhaseIncrement = 0.0;
double mInputPhase = 0.0;
double mOutputPhase = 0.0;
double mMagnitude = 0.0;
int32_t mFramesAccumulated = 0;
double mSinAccumulator = 0.0;
double mCosAccumulator = 0.0;
float mMaxGlitchDelta = 0.0f;
double mMaxGlitchDelta = 0.0;
int32_t mGlitchCount = 0;
int32_t mNonGlitchCount = 0;
int32_t mConsecutiveBadFrames = 0;
int32_t mConsecutiveGoodFrames = 0;
int32_t mGlitchLength = 0;
float mScaledTolerance = 0.0;
int mDownCounter = IDLE_FRAME_COUNT;
int32_t mFrameCounter = 0;
float mOutputAmplitude = 0.75;
int32_t mForceGlitchDuration = 0; // if > 0 then force a glitch for debugging
int32_t mForceGlitchCounter = 4 * 48000; // count down and trigger at zero
int32_t mForceGlitchDurationFrames = 0; // if > 0 then force a glitch for debugging
static constexpr int32_t kForceGlitchPeriod = 2 * 48000; // How often we glitch
static constexpr float kForceGlitchOffset = 0.20f;
int32_t mForceGlitchCounter = kForceGlitchPeriod; // count down and trigger at zero
// measure background noise continuously as a deviation from the expected signal
double mSumSquareSignal = 0.0;
@@ -425,14 +410,7 @@ private:
double mMeanSquareNoise = 0.0;
PeakDetector mPeakFollower;
PseudoRandom mWhiteNoise;
sine_state_t mState = STATE_IDLE;
InfiniteRecording<float> mInfiniteRecording;
int64_t mLastGlitchPosition;
};
#endif //OBOETESTER_GLITCHANALYZER_H
#endif //ANALYZER_GLITCH_ANALYZER_H
@@ -34,6 +34,7 @@ public:
int32_t readFrom(T *buffer, size_t position, size_t count) {
const size_t maxPosition = mWritten.load();
position = std::min(position, maxPosition);
size_t numToRead = std::min(count, mMaxSamples);
numToRead = std::min(numToRead, maxPosition - position);
if (numToRead == 0) return 0;
@@ -61,7 +62,7 @@ public:
private:
std::unique_ptr<T[]> mData;
std::atomic<size_t> mWritten{0};
const size_t mMaxSamples;
std::atomic<size_t> mWritten{0};
const size_t mMaxSamples;
};
#endif //OBOETESTER_INFINITE_RECORDING_H
@@ -25,45 +25,61 @@
#include <algorithm>
#include <assert.h>
#include <cctype>
#include <iomanip>
#include <iostream>
#include <math.h>
#include <memory>
#include <sstream>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <vector>
#include "RandomPulseGenerator.h"
#include "PeakDetector.h"
#include "PseudoRandom.h"
#include "RandomPulseGenerator.h"
// This is used when the code is in not in Android.
#ifndef ALOGD
#define ALOGD LOGD
#define ALOGE LOGE
#define ALOGW LOGW
#endif
#define LOOPBACK_RESULT_TAG "RESULT: "
static constexpr int32_t kDefaultSampleRate = 48000;
static constexpr int32_t kMillisPerSecond = 1000;
static constexpr int32_t kMaxLatencyMillis = 700; // arbitrary and generous
static constexpr double kMinimumConfidence = 0.2;
// Enable or disable the optimized latency calculation.
#define USE_FAST_LATENCY_CALCULATION 1
typedef struct LatencyReport_s {
int32_t latencyInFrames = 0.0;
double confidence = 0.0;
static constexpr int32_t kDefaultSampleRate = 48000;
static constexpr int32_t kMillisPerSecond = 1000; // by definition
static constexpr int32_t kMaxLatencyMillis = 1000; // arbitrary and generous
struct LatencyReport {
int32_t latencyInFrames = 0;
double correlation = 0.0;
void reset() {
latencyInFrames = 0;
confidence = 0.0;
correlation = 0.0;
}
} LatencyReport;
};
// Calculate a normalized cross correlation.
static double calculateNormalizedCorrelation(const float *a,
/**
* Calculate a normalized cross correlation.
* @return value between -1.0 and 1.0
*/
static float calculateNormalizedCorrelation(const float *a,
const float *b,
int windowSize)
{
double correlation = 0.0;
double sumProducts = 0.0;
double sumSquares = 0.0;
int windowSize,
int stride) {
float correlation = 0.0;
float sumProducts = 0.0;
float sumSquares = 0.0;
// Correlate a against b.
for (int i = 0; i < windowSize; i++) {
for (int i = 0; i < windowSize; i += stride) {
float s1 = a[i];
float s2 = b[i];
// Use a normalized cross-correlation.
@@ -72,7 +88,7 @@ static double calculateNormalizedCorrelation(const float *a,
}
if (sumSquares >= 1.0e-9) {
correlation = (float) (2.0 * sumProducts / sumSquares);
correlation = 2.0 * sumProducts / sumSquares;
}
return correlation;
}
@@ -80,7 +96,7 @@ static double calculateNormalizedCorrelation(const float *a,
static double calculateRootMeanSquare(float *data, int32_t numSamples) {
double sum = 0.0;
for (int32_t i = 0; i < numSamples; i++) {
float sample = data[i];
double sample = data[i];
sum += sample * sample;
}
return sqrt(sum / numSamples);
@@ -88,24 +104,20 @@ static double calculateRootMeanSquare(float *data, int32_t numSamples) {
/**
* Monophonic recording with processing.
* Samples are stored as floats internally.
*/
class AudioRecording
{
public:
AudioRecording() {
}
~AudioRecording() {
delete[] mData;
}
void allocate(int maxFrames) {
delete[] mData;
mData = new float[maxFrames];
mData = std::make_unique<float[]>(maxFrames);
mMaxFrames = maxFrames;
mFrameCounter = 0;
}
// Write SHORT data from the first channel.
int32_t write(int16_t *inputData, int32_t inputChannelCount, int32_t numFrames) {
int32_t write(const int16_t *inputData, int32_t inputChannelCount, int32_t numFrames) {
// stop at end of buffer
if ((mFrameCounter + numFrames) > mMaxFrames) {
numFrames = mMaxFrames - mFrameCounter;
@@ -117,7 +129,7 @@ public:
}
// Write FLOAT data from the first channel.
int32_t write(float *inputData, int32_t inputChannelCount, int32_t numFrames) {
int32_t write(const float *inputData, int32_t inputChannelCount, int32_t numFrames) {
// stop at end of buffer
if ((mFrameCounter + numFrames) > mMaxFrames) {
numFrames = mMaxFrames - mFrameCounter;
@@ -128,35 +140,37 @@ public:
return numFrames;
}
// Write FLOAT data from the first channel.
// Write single FLOAT value.
int32_t write(float sample) {
// stop at end of buffer
if (mFrameCounter < mMaxFrames) {
mData[mFrameCounter++] = sample;
return 1;
}
return 1;
return 0;
}
void clear() {
mFrameCounter = 0;
}
int32_t size() {
int32_t size() const {
return mFrameCounter;
}
bool isFull() {
bool isFull() const {
return mFrameCounter >= mMaxFrames;
}
float *getData() {
return mData;
float *getData() const {
return mData.get();
}
void setSampleRate(int32_t sampleRate) {
mSampleRate = sampleRate;
}
int32_t getSampleRate() {
int32_t getSampleRate() const {
return mSampleRate;
}
@@ -164,9 +178,24 @@ public:
* Square the samples so they are all positive and so the peaks are emphasized.
*/
void square() {
float *x = mData.get();
for (int i = 0; i < mFrameCounter; i++) {
const float sample = mData[i];
mData[i] = sample * sample;
x[i] *= x[i];
}
}
// Envelope follower that rides over the peak values.
void detectPeaks(float decay) {
float level = 0.0f;
float *x = mData.get();
for (int i = 0; i < mFrameCounter; i++) {
level *= decay; // exponential decay
float input = fabs(x[i]);
// never fall below the input signal
if (input > level) {
level = input;
}
x[i] = level; // write result back into the array
}
}
@@ -179,7 +208,7 @@ public:
float normalize(float target) {
float maxValue = 1.0e-9f;
for (int i = 0; i < mFrameCounter; i++) {
maxValue = std::max(maxValue, abs(mData[i]));
maxValue = std::max(maxValue, fabsf(mData[i]));
}
float gain = target / maxValue;
for (int i = 0; i < mFrameCounter; i++) {
@@ -189,62 +218,127 @@ public:
}
private:
float *mData = nullptr;
std::unique_ptr<float[]> mData;
int32_t mFrameCounter = 0;
int32_t mMaxFrames = 0;
int32_t mSampleRate = kDefaultSampleRate; // common default
};
static int measureLatencyFromPulse(AudioRecording &recorded,
AudioRecording &pulse,
int32_t framesPerEncodedBit,
LatencyReport *report) {
/**
* Find latency using cross correlation in window of the recorded audio.
* The stride is used to skip over samples and reduce the CPU load.
*/
static int measureLatencyFromPulsePartial(AudioRecording &recorded,
int32_t recordedOffset,
int32_t recordedWindowSize,
AudioRecording &pulse,
LatencyReport *report,
int32_t stride) {
report->reset();
report->latencyInFrames = 0;
report->confidence = 0.0;
if (recordedOffset + recordedWindowSize + pulse.size() > recorded.size()) {
ALOGE("%s() tried to correlate past end of recording, recordedOffset = %d frames\n",
__func__, recordedOffset);
return -3;
}
int numCorrelations = recorded.size() - pulse.size();
int32_t numCorrelations = recordedWindowSize / stride;
if (numCorrelations < 10) {
LOGE("%s() recording too small = %d frames", __func__, recorded.size());
ALOGE("%s() recording too small = %d frames, numCorrelations = %d\n",
__func__, recorded.size(), numCorrelations);
return -1;
}
std::unique_ptr<float[]> correlations= std::make_unique<float[]>(numCorrelations);
// Correlate pulse against the recorded data.
for (int i = 0; i < numCorrelations; i++) {
float correlation = (float) calculateNormalizedCorrelation(&recorded.getData()[i],
&pulse.getData()[0],
pulse.size());
for (int32_t i = 0; i < numCorrelations; i++) {
const int32_t index = (i * stride) + recordedOffset;
float correlation = calculateNormalizedCorrelation(&recorded.getData()[index],
&pulse.getData()[0],
pulse.size(),
stride);
correlations[i] = correlation;
}
// Find highest peak in correlation array.
float peakCorrelation = 0.0;
int peakIndex = -1;
for (int i = 0; i < numCorrelations; i++) {
float value = abs(correlations[i]);
int32_t peakIndex = -1;
for (int32_t i = 0; i < numCorrelations; i++) {
float value = fabsf(correlations[i]);
if (value > peakCorrelation) {
peakCorrelation = value;
peakIndex = i;
}
}
if (peakIndex < 0) {
LOGE("%s() no signal for correlation", __func__);
ALOGE("%s() no signal for correlation\n", __func__);
return -2;
}
#if 0
// Dump correlation data for charting.
else {
const int32_t margin = 50;
int32_t startIndex = std::max(0, peakIndex - margin);
int32_t endIndex = std::min(numCorrelations - 1, peakIndex + margin);
for (int32_t index = startIndex; index < endIndex; index++) {
ALOGD("Correlation, %d, %f", index, correlations[index]);
}
}
#endif
report->latencyInFrames = peakIndex;
report->confidence = peakCorrelation;
report->latencyInFrames = recordedOffset + (peakIndex * stride);
report->correlation = peakCorrelation;
return 0;
}
#if USE_FAST_LATENCY_CALCULATION
static int measureLatencyFromPulse(AudioRecording &recorded,
AudioRecording &pulse,
LatencyReport *report) {
const int32_t coarseStride = 16;
const int32_t fineWindowSize = coarseStride * 8;
const int32_t fineStride = 1;
LatencyReport courseReport;
courseReport.reset();
// Do a rough search, skipping over most of the samples.
int result = measureLatencyFromPulsePartial(recorded,
0, // recordedOffset,
recorded.size() - pulse.size(),
pulse,
&courseReport,
coarseStride);
if (result != 0) {
return result;
}
// Now do a fine resolution search near the coarse latency result.
int32_t recordedOffset = std::max(0, courseReport.latencyInFrames - (fineWindowSize / 2));
result = measureLatencyFromPulsePartial(recorded,
recordedOffset,
fineWindowSize,
pulse,
report,
fineStride );
return result;
}
#else
// TODO - When we are confident of the new code we can remove this old code.
static int measureLatencyFromPulse(AudioRecording &recorded,
AudioRecording &pulse,
LatencyReport *report) {
return measureLatencyFromPulsePartial(recorded,
0,
recorded.size() - pulse.size(),
pulse,
report,
1 );
}
#endif
// ====================================================================================
class LoopbackProcessor {
public:
virtual ~LoopbackProcessor() = default;
// Note that these values must match the switch in RoundTripLatencyActivity.h
enum result_code {
RESULT_OK = 0,
ERROR_NOISY = -99,
@@ -256,7 +350,7 @@ public:
ERROR_NO_LOCK
};
virtual void onStartTest() {
virtual void prepareToTest() {
reset();
}
@@ -265,11 +359,11 @@ public:
mResetCount++;
}
virtual result_code processInputFrame(float *frameData, int channelCount) = 0;
virtual result_code processInputFrame(const float *frameData, int channelCount) = 0;
virtual result_code processOutputFrame(float *frameData, int channelCount) = 0;
void process(float *inputData, int inputChannelCount, int numInputFrames,
float *outputData, int outputChannelCount, int numOutputFrames) {
void process(const float *inputData, int inputChannelCount, int numInputFrames,
float *outputData, int outputChannelCount, int numOutputFrames) {
int numBoth = std::min(numInputFrames, numOutputFrames);
// Process one frame at a time.
for (int i = 0; i < numBoth; i++) {
@@ -290,7 +384,7 @@ public:
}
}
virtual void analyze() = 0;
virtual std::string analyze() = 0;
virtual void printStatus() {};
@@ -320,11 +414,11 @@ public:
mSampleRate = sampleRate;
}
int32_t getSampleRate() {
int32_t getSampleRate() const {
return mSampleRate;
}
int32_t getResetCount() {
int32_t getResetCount() const {
return mResetCount;
}
@@ -334,10 +428,41 @@ public:
reset();
}
/**
* Some analyzers may only look at one channel.
* You can optionally specify that channel here.
*
* @param inputChannel
*/
void setInputChannel(int inputChannel) {
mInputChannel = inputChannel;
}
int getInputChannel() const {
return mInputChannel;
}
/**
* Some analyzers may only generate one channel.
* You can optionally specify that channel here.
*
* @param outputChannel
*/
void setOutputChannel(int outputChannel) {
mOutputChannel = outputChannel;
}
int getOutputChannel() const {
return mOutputChannel;
}
protected:
int32_t mResetCount = 0;
private:
int32_t mInputChannel = 0;
int32_t mOutputChannel = 0;
int32_t mSampleRate = kDefaultSampleRate;
int32_t mResult = 0;
};
@@ -348,19 +473,44 @@ public:
LatencyAnalyzer() : LoopbackProcessor() {}
virtual ~LatencyAnalyzer() = default;
virtual int32_t getProgress() = 0;
/**
* Call this after the constructor because it calls other virtual methods.
*/
virtual void setup() = 0;
virtual int getState() = 0;
virtual int32_t getProgress() const = 0;
virtual int getState() const = 0;
// @return latency in frames
virtual int32_t getMeasuredLatency() = 0;
virtual int32_t getMeasuredLatency() const = 0;
virtual double getMeasuredConfidence() = 0;
/**
* This is an overall confidence in the latency result based on correlation, SNR, etc.
* @return probability value between 0.0 and 1.0
*/
double getMeasuredConfidence() const {
// Limit the ratio and prevent divide-by-zero.
double noiseSignalRatio = getSignalRMS() <= getBackgroundRMS()
? 1.0 : getBackgroundRMS() / getSignalRMS();
// Prevent high background noise and low signals from generating false matches.
double adjustedConfidence = getMeasuredCorrelation() - noiseSignalRatio;
return std::max(0.0, adjustedConfidence);
}
virtual double getBackgroundRMS() = 0;
/**
* Cross correlation value for the noise pulse against
* the corresponding position in the normalized recording.
*
* @return value between -1.0 and 1.0
*/
virtual double getMeasuredCorrelation() const = 0;
virtual double getSignalRMS() = 0;
virtual double getBackgroundRMS() const = 0;
virtual double getSignalRMS() const = 0;
virtual bool hasEnoughData() const = 0;
};
// ====================================================================================
@@ -374,27 +524,15 @@ public:
class PulseLatencyAnalyzer : public LatencyAnalyzer {
public:
PulseLatencyAnalyzer() : LatencyAnalyzer() {
void setup() override {
int32_t pulseLength = calculatePulseLength();
int32_t maxLatencyFrames = getSampleRate() * kMaxLatencyMillis / kMillisPerSecond;
int32_t numPulseBits = getSampleRate() * kPulseLengthMillis
/ (kFramesPerEncodedBit * kMillisPerSecond);
int32_t pulseLength = numPulseBits * kFramesPerEncodedBit;
mFramesToRecord = pulseLength + maxLatencyFrames;
LOGD("PulseLatencyAnalyzer: allocate recording with %d frames", mFramesToRecord);
mAudioRecording.allocate(mFramesToRecord);
mAudioRecording.setSampleRate(getSampleRate());
generateRandomPulse(pulseLength);
}
void generateRandomPulse(int32_t pulseLength) {
mPulse.allocate(pulseLength);
RandomPulseGenerator pulser(kFramesPerEncodedBit);
for (int i = 0; i < pulseLength; i++) {
mPulse.write(pulser.nextFloat());
}
}
int getState() override {
int getState() const override {
return mState;
}
@@ -405,7 +543,8 @@ public:
void reset() override {
LoopbackProcessor::reset();
mDownCounter = getSampleRate() / 2;
mState = STATE_MEASURE_BACKGROUND;
mDownCounter = (int32_t) (getSampleRate() * kBackgroundMeasurementLengthSeconds);
mLoopCounter = 0;
mPulseCursor = 0;
@@ -414,13 +553,12 @@ public:
mBackgroundRMS = 0.0f;
mSignalRMS = 0.0f;
LOGD("state reset to STATE_MEASURE_BACKGROUND");
mState = STATE_MEASURE_BACKGROUND;
generatePulseRecording(calculatePulseLength());
mAudioRecording.clear();
mLatencyReport.reset();
}
bool hasEnoughData() {
bool hasEnoughData() const override {
return mAudioRecording.isFull();
}
@@ -428,96 +566,103 @@ public:
return mState == STATE_DONE;
}
int32_t getProgress() override {
int32_t getProgress() const override {
return mAudioRecording.size();
}
void analyze() override {
LOGD("PulseLatencyAnalyzer ---------------");
LOGD(LOOPBACK_RESULT_TAG "test.state = %8d", mState);
LOGD(LOOPBACK_RESULT_TAG "test.state.name = %8s", convertStateToText(mState));
LOGD(LOOPBACK_RESULT_TAG "background.rms = %8f", mBackgroundRMS);
std::string analyze() override {
std::stringstream report;
report << "PulseLatencyAnalyzer ---------------\n";
report << LOOPBACK_RESULT_TAG "test.state = "
<< std::setw(8) << mState << "\n";
report << LOOPBACK_RESULT_TAG "test.state.name = "
<< convertStateToText(mState) << "\n";
report << LOOPBACK_RESULT_TAG "background.rms = "
<< std::setw(8) << mBackgroundRMS << "\n";
int32_t newResult = RESULT_OK;
if (mState != STATE_GOT_DATA) {
LOGD("WARNING - Bad state. Check volume on device.");
report << "WARNING - Bad state. Check volume on device.\n";
// setResult(ERROR_INVALID_STATE);
} else {
LOGD("Please wait several seconds for cross-correlation to complete.");
float gain = mAudioRecording.normalize(1.0f);
measureLatencyFromPulse(mAudioRecording,
mPulse,
kFramesPerEncodedBit,
&mLatencyReport);
measureLatency();
if (mLatencyReport.confidence < kMinimumConfidence) {
LOGD(" ERROR - confidence too low!");
// Calculate signalRMS even if it is bogus.
// Also it may be used in the confidence calculation below.
mSignalRMS = calculateRootMeanSquare(
&mAudioRecording.getData()[mLatencyReport.latencyInFrames], mPulse.size())
/ gain;
if (getMeasuredConfidence() < getMinimumConfidence()) {
report << " ERROR - confidence too low!";
newResult = ERROR_CONFIDENCE;
} else {
mSignalRMS = calculateRootMeanSquare(
&mAudioRecording.getData()[mLatencyReport.latencyInFrames], mPulse.size())
/ gain;
}
#if OBOE_ENABLE_LOGGING
double latencyMillis = kMillisPerSecond * (double) mLatencyReport.latencyInFrames
/ getSampleRate();
#endif
LOGD(LOOPBACK_RESULT_TAG "latency.frames = %8d",
mLatencyReport.latencyInFrames);
LOGD(LOOPBACK_RESULT_TAG "latency.msec = %8.2f",
latencyMillis);
LOGD(LOOPBACK_RESULT_TAG "latency.confidence = %8.6f",
mLatencyReport.confidence);
report << LOOPBACK_RESULT_TAG "latency.frames = " << std::setw(8)
<< mLatencyReport.latencyInFrames << "\n";
report << LOOPBACK_RESULT_TAG "latency.msec = " << std::setw(8)
<< latencyMillis << "\n";
report << LOOPBACK_RESULT_TAG "latency.confidence = " << std::setw(8)
<< getMeasuredConfidence() << "\n";
report << LOOPBACK_RESULT_TAG "latency.correlation = " << std::setw(8)
<< getMeasuredCorrelation() << "\n";
}
mState = STATE_DONE;
if (getResult() == RESULT_OK) {
setResult(newResult);
}
return report.str();
}
int32_t getMeasuredLatency() override {
int32_t getMeasuredLatency() const override {
return mLatencyReport.latencyInFrames;
}
double getMeasuredConfidence() override {
return mLatencyReport.confidence;
double getMeasuredCorrelation() const override {
return mLatencyReport.correlation;
}
double getBackgroundRMS() override {
double getBackgroundRMS() const override {
return mBackgroundRMS;
}
double getSignalRMS() override {
double getSignalRMS() const override {
return mSignalRMS;
}
void printStatus() override {
LOGD("st = %d", mState);
bool isRecordingComplete() {
return mState == STATE_GOT_DATA;
}
result_code processInputFrame(float *frameData, int channelCount) override {
void printStatus() override {
ALOGD("latency: st = %d = %s", mState, convertStateToText(mState));
}
result_code processInputFrame(const float *frameData, int /* channelCount */) override {
echo_state nextState = mState;
mLoopCounter++;
float input = frameData[0];
switch (mState) {
case STATE_MEASURE_BACKGROUND:
// Measure background RMS on channel 0
mBackgroundSumSquare += frameData[0] * frameData[0];
mBackgroundSumSquare += static_cast<double>(input) * input;
mBackgroundSumCount++;
mDownCounter--;
if (mDownCounter <= 0) {
mBackgroundRMS = sqrtf(mBackgroundSumSquare / mBackgroundSumCount);
nextState = STATE_IN_PULSE;
mPulseCursor = 0;
LOGD("LatencyAnalyzer state => STATE_SENDING_PULSE");
}
break;
case STATE_IN_PULSE:
// Record input until the mAudioRecording is full.
mAudioRecording.write(frameData, channelCount, 1);
mAudioRecording.write(input);
if (hasEnoughData()) {
LOGD("LatencyAnalyzer state => STATE_GOT_DATA");
nextState = STATE_GOT_DATA;
}
break;
@@ -560,6 +705,27 @@ public:
return RESULT_OK;
}
protected:
virtual int32_t calculatePulseLength() const = 0;
virtual void generatePulseRecording(int32_t pulseLength) = 0;
virtual void measureLatency() = 0;
virtual double getMinimumConfidence() const {
return 0.5;
}
AudioRecording mPulse;
AudioRecording mAudioRecording; // contains only the input after starting the pulse
LatencyReport mLatencyReport;
static constexpr int32_t kPulseLengthMillis = 500;
float mPulseAmplitude = 0.5f;
double mBackgroundRMS = 0.0;
double mSignalRMS = 0.0;
private:
enum echo_state {
@@ -570,42 +736,127 @@ private:
};
const char *convertStateToText(echo_state state) {
const char *result = "Unknown";
switch(state) {
switch (state) {
case STATE_MEASURE_BACKGROUND:
result = "INIT";
break;
return "INIT";
case STATE_IN_PULSE:
result = "PULSE";
break;
return "PULSE";
case STATE_GOT_DATA:
result = "GOT_DATA";
break;
return "GOT_DATA";
case STATE_DONE:
result = "DONE";
break;
return "DONE";
}
return result;
return "UNKNOWN";
}
int32_t mDownCounter = 500;
int32_t mLoopCounter = 0;
echo_state mState = STATE_MEASURE_BACKGROUND;
static constexpr int32_t kFramesPerEncodedBit = 8; // multiple of 2
static constexpr int32_t kPulseLengthMillis = 500;
static constexpr double kBackgroundMeasurementLengthSeconds = 0.5;
AudioRecording mPulse;
int32_t mPulseCursor = 0;
float mBackgroundSumSquare = 0.0f;
double mBackgroundSumSquare = 0.0;
int32_t mBackgroundSumCount = 0;
float mBackgroundRMS = 0.0f;
float mSignalRMS = 0.0f;
int32_t mFramesToRecord = 0;
AudioRecording mAudioRecording; // contains only the input after starting the pulse
LatencyReport mLatencyReport;
};
/**
* This algorithm uses a series of random bits encoded using the
* Manchester encoder. It works well for wired loopback but not very well for
* through the air loopback.
*/
class EncodedRandomLatencyAnalyzer : public PulseLatencyAnalyzer {
protected:
int32_t calculatePulseLength() const override {
// Calculate integer number of bits.
int32_t numPulseBits = getSampleRate() * kPulseLengthMillis
/ (kFramesPerEncodedBit * kMillisPerSecond);
return numPulseBits * kFramesPerEncodedBit;
}
void generatePulseRecording(int32_t pulseLength) override {
mPulse.allocate(pulseLength);
RandomPulseGenerator pulser(kFramesPerEncodedBit);
for (int i = 0; i < pulseLength; i++) {
mPulse.write(pulser.nextFloat() * mPulseAmplitude);
}
}
double getMinimumConfidence() const override {
return 0.2;
}
void measureLatency() override {
measureLatencyFromPulse(mAudioRecording,
mPulse,
&mLatencyReport);
}
private:
static constexpr int32_t kFramesPerEncodedBit = 8; // multiple of 2
};
/**
* This algorithm uses White Noise sent in a short burst pattern.
* The original signal and the recorded signal are then run through
* an envelope follower to convert the fine detail into more of
* a rectangular block before the correlation phase.
*/
class WhiteNoiseLatencyAnalyzer : public PulseLatencyAnalyzer {
protected:
int32_t calculatePulseLength() const override {
return getSampleRate() * kPulseLengthMillis / kMillisPerSecond;
}
void generatePulseRecording(int32_t pulseLength) override {
mPulse.allocate(pulseLength);
// Turn the noise on and off to sharpen the correlation peak.
// Use more zeros than ones so that the correlation will be less than 0.5 even when there
// is a strong background noise.
int8_t pattern[] = {1, 0, 0,
1, 1, 0, 0, 0,
1, 1, 1, 0, 0, 0, 0,
1, 1, 1, 1, 0, 0, 0, 0, 0
};
PseudoRandom random;
const int32_t numSections = sizeof(pattern);
const int32_t framesPerSection = pulseLength / numSections;
for (int section = 0; section < numSections; section++) {
if (pattern[section]) {
for (int i = 0; i < framesPerSection; i++) {
mPulse.write((float) (random.nextRandomDouble() * mPulseAmplitude));
}
} else {
for (int i = 0; i < framesPerSection; i++) {
mPulse.write(0.0f);
}
}
}
// Write any remaining frames.
int32_t framesWritten = framesPerSection * numSections;
for (int i = framesWritten; i < pulseLength; i++) {
mPulse.write(0.0f);
}
}
void measureLatency() override {
// Smooth out the noise so we see rectangular blocks.
// This improves immunity against phase cancellation and distortion.
static constexpr float decay = 0.99f; // just under 1.0, lower numbers decay faster
mAudioRecording.detectPeaks(decay);
mPulse.detectPeaks(decay);
measureLatencyFromPulse(mAudioRecording,
mPulse,
&mLatencyReport);
}
};
#endif // ANALYZER_LATENCY_ANALYZER_H
@@ -41,15 +41,17 @@ public:
, mCursor(samplesPerPulse) {
}
virtual ~ManchesterEncoder() = default;
/**
* This will be called when the next byte is needed.
* @return
* @return next byte
*/
virtual uint8_t onNextByte() = 0;
/**
* Generate the next floating point sample.
* @return
* @return next float
*/
virtual float nextFloat() {
advanceSample();
@@ -64,10 +66,9 @@ protected:
/**
* This will be called when a new bit is ready to be encoded.
* It can be used to prepare the encoded samples.
* @param current
*/
virtual void onNextBit(bool current) {};
virtual void onNextBit(bool /* current */) {};
void advanceSample() {
// Are we ready for a new bit?
if (++mCursor >= mSamplesPerPulse) {
@@ -19,6 +19,11 @@
#include <math.h>
/**
* Measure a peak envelope by rising with the peaks,
* and decaying exponentially after each peak.
* The absolute value of the input signal is used.
*/
class PeakDetector {
public:
@@ -27,20 +32,35 @@ public:
}
double process(double input) {
mLevel *= mDecay;
mLevel *= mDecay; // exponential decay
input = fabs(input);
// never fall below the input signal
if (input > mLevel) {
mLevel = input;
}
return mLevel;
}
double getLevel() {
double getLevel() const {
return mLevel;
}
double getDecay() const {
return mDecay;
}
/**
* Multiply the level by this amount on every iteration.
* This provides an exponential decay curve.
* A value just under 1.0 is best, for example, 0.99;
* @param decay scale level for each input
*/
void setDecay(double decay) {
mDecay = decay;
}
private:
static constexpr float kDefaultDecay = 0.99f;
static constexpr double kDefaultDecay = 0.99f;
double mLevel = 0.0;
double mDecay = kDefaultDecay;
@@ -22,8 +22,7 @@
class PseudoRandom {
public:
PseudoRandom() {}
PseudoRandom(int64_t seed)
PseudoRandom(int64_t seed = 99887766)
: mSeed(seed)
{}
@@ -36,7 +35,8 @@ public:
return nextRandomInteger() * (0.5 / (((int32_t)1) << 30));
}
/** Calculate random 32 bit number using linear-congruential method.
/** Calculate random 32 bit number using linear-congruential method
* with known real-time performance.
*/
int32_t nextRandomInteger() {
#if __has_builtin(__builtin_mul_overflow) && __has_builtin(__builtin_add_overflow)
@@ -51,7 +51,7 @@ public:
}
private:
int64_t mSeed = 99887766;
int64_t mSeed;
};
#endif //ANALYZER_PSEUDORANDOM_H
@@ -29,12 +29,14 @@ public:
: RoundedManchesterEncoder(samplesPerPulse) {
}
virtual ~RandomPulseGenerator() = default;
/**
* This will be called when the next byte is needed.
* @return random byte
*/
uint8_t onNextByte() override {
return static_cast<uint8_t>(rand() & 0x00FF);
return static_cast<uint8_t>(rand());
}
};
@@ -35,30 +35,30 @@ public:
mZeroAfterZero = std::make_unique<float[]>(samplesPerPulse);
mZeroAfterOne = std::make_unique<float[]>(samplesPerPulse);
int i = 0;
for (int j = 0; j < rampSize; j++) {
float phase = (j + 1) * M_PI / rampSize;
int sampleIndex = 0;
for (int rampIndex = 0; rampIndex < rampSize; rampIndex++) {
float phase = (rampIndex + 1) * M_PI / rampSize;
float sample = -cosf(phase);
mZeroAfterZero[i] = sample;
mZeroAfterOne[i] = 1.0f;
i++;
mZeroAfterZero[sampleIndex] = sample;
mZeroAfterOne[sampleIndex] = 1.0f;
sampleIndex++;
}
for (int j = 0; j < rampSize; j++) {
mZeroAfterZero[i] = 1.0f;
mZeroAfterOne[i] = 1.0f;
i++;
for (int rampIndex = 0; rampIndex < rampSize; rampIndex++) {
mZeroAfterZero[sampleIndex] = 1.0f;
mZeroAfterOne[sampleIndex] = 1.0f;
sampleIndex++;
}
for (int j = 0; j < rampSize; j++) {
float phase = (j + 1) * M_PI / rampSize;
for (int rampIndex = 0; rampIndex < rampSize; rampIndex++) {
float phase = (rampIndex + 1) * M_PI / rampSize;
float sample = cosf(phase);
mZeroAfterZero[i] = sample;
mZeroAfterOne[i] = sample;
i++;
mZeroAfterZero[sampleIndex] = sample;
mZeroAfterOne[sampleIndex] = sample;
sampleIndex++;
}
for (int j = 0; j < rampSize; j++) {
mZeroAfterZero[i] = -1.0f;
mZeroAfterOne[i] = -1.0f;
i++;
for (int rampIndex = 0; rampIndex < rampSize; rampIndex++) {
mZeroAfterZero[sampleIndex] = -1.0f;
mZeroAfterOne[sampleIndex] = -1.0f;
sampleIndex++;
}
}
@@ -70,7 +70,6 @@ public:
mPreviousBit = current;
}
float nextFloat() override {
advanceSample();
float output = mCurrentSamples[mCursor];
@@ -32,4 +32,4 @@ int32_t ExponentialShape::onProcess(int32_t numFrames) {
}
return numFrames;
}
}
@@ -25,7 +25,7 @@
*
* The waveform is not band-limited so it will have aliasing artifacts at higher frequencies.
*/
class ExponentialShape : public flowgraph::FlowGraphFilter {
class ExponentialShape : public oboe::flowgraph::FlowGraphFilter {
public:
ExponentialShape();
@@ -17,7 +17,7 @@
#include "LinearShape.h"
using namespace flowgraph;
using namespace oboe::flowgraph;
LinearShape::LinearShape()
: FlowGraphFilter(1) {
@@ -33,4 +33,4 @@ int32_t LinearShape::onProcess(int numFrames) {
}
return numFrames;
}
}
@@ -23,7 +23,7 @@
/**
* Convert an input between -1.0 and +1.0 to a linear region between min and max.
*/
class LinearShape : public flowgraph::FlowGraphFilter {
class LinearShape : public oboe::flowgraph::FlowGraphFilter {
public:
LinearShape();
@@ -16,7 +16,7 @@
#include "OscillatorBase.h"
using namespace flowgraph;
using namespace oboe::flowgraph;
OscillatorBase::OscillatorBase()
: frequency(*this, 1)
@@ -29,7 +29,7 @@
* This module has "frequency" and "amplitude" ports for control.
*/
class OscillatorBase : public flowgraph::FlowGraphNode {
class OscillatorBase : public oboe::flowgraph::FlowGraphNode {
public:
OscillatorBase();
@@ -61,16 +61,16 @@ public:
/**
* Control the frequency of the oscillator in Hz.
*/
flowgraph::FlowGraphPortFloatInput frequency;
oboe::flowgraph::FlowGraphPortFloatInput frequency;
/**
* Control the linear amplitude of the oscillator.
* Silence is 0.0.
* A typical full amplitude would be 1.0.
*/
flowgraph::FlowGraphPortFloatInput amplitude;
oboe::flowgraph::FlowGraphPortFloatInput amplitude;
flowgraph::FlowGraphPortFloatOutput output;
oboe::flowgraph::FlowGraphPortFloatOutput output;
protected:
/**
@@ -0,0 +1,32 @@
/*
* Copyright 2022 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#include <math.h>
#include <unistd.h>
#include "WhiteNoise.h"
int32_t WhiteNoise::onProcess(int32_t numFrames) {
const float *amplitudes = amplitude.getBuffer();
float *buffer = output.getBuffer();
for (int i = 0; i < numFrames; i++) {
float noise = (float) mPseudoRandom.nextRandomDouble(); // -1 to +1
*buffer++ = noise * (*amplitudes++);
}
return numFrames;
}
@@ -0,0 +1,57 @@
/*
* Copyright 2022 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#ifndef FLOWGRAPH_WHITE_NOISE_H
#define FLOWGRAPH_WHITE_NOISE_H
#include <unistd.h>
#include "flowgraph/FlowGraphNode.h"
#include "../analyzer/PseudoRandom.h"
/**
* White noise with equal energy in all frequencies up to the Nyquist.
* This is a based on random numbers with a uniform distribution.
*/
class WhiteNoise : public oboe::flowgraph::FlowGraphNode {
public:
WhiteNoise()
: oboe::flowgraph::FlowGraphNode()
, amplitude(*this, 1)
, output(*this, 1)
{
}
virtual ~WhiteNoise() = default;
int32_t onProcess(int32_t numFrames) override;
/**
* Control the amplitude amplitude of the noise.
* Silence is 0.0.
* A typical full amplitude would be 1.0.
*/
oboe::flowgraph::FlowGraphPortFloatInput amplitude;
oboe::flowgraph::FlowGraphPortFloatOutput output;
private:
PseudoRandom mPseudoRandom;
};
#endif //FLOWGRAPH_WHITE_NOISE_H
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,163 @@
/*
* Copyright (C) 2016 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
* This code was translated from the JSyn Java code.
* JSyn is Copyright 2009 Phil Burk, Mobileer Inc
* JSyn is licensed under the Apache License, Version 2.0
*/
#ifndef SYNTHMARK_BIQUAD_FILTER_H
#define SYNTHMARK_BIQUAD_FILTER_H
#include <cstdint>
#include <math.h>
#include "SynthTools.h"
#include "UnitGenerator.h"
namespace marksynth {
#define BIQUAD_MIN_FREQ (0.00001f) // REVIEW
#define BIQUAD_MIN_Q (0.00001f) // REVIEW
#define RECALCULATE_PER_SAMPLE 0
/**
* Time varying lowpass resonant filter.
*/
class BiquadFilter : public UnitGenerator
{
public:
BiquadFilter()
: mQ(1.0)
{
xn1 = xn2 = yn1 = yn2 = (synth_float_t) 0;
a0 = a1 = a2 = b1 = b2 = (synth_float_t) 0;
}
virtual ~BiquadFilter() = default;
/**
* Resonance, typically between 1.0 and 10.0.
* Input will clipped at a BIQUAD_MIN_Q.
*/
void setQ(synth_float_t q) {
if( q < BIQUAD_MIN_Q ) {
q = BIQUAD_MIN_Q;
}
mQ = q;
}
synth_float_t getQ() {
return mQ;
}
void generate(synth_float_t *input,
synth_float_t *frequencies,
int32_t numSamples) {
synth_float_t xn, yn;
#if RECALCULATE_PER_SAMPLE == 0
calculateCoefficients(frequencies[0], mQ);
#endif
for (int i = 0; i < numSamples; i++) {
#if RECALCULATE_PER_SAMPLE == 1
calculateCoefficients(frequencies[i], mQ);
#endif
// Generate outputs by filtering inputs.
xn = input[i];
synth_float_t finite = (a0 * xn) + (a1 * xn1) + (a2 * xn2);
// Use double precision for recursive portion.
yn = finite - (b1 * yn1) - (b2 * yn2);
output[i] = (synth_float_t) yn;
// Delay input and output values.
xn2 = xn1;
xn1 = xn;
yn2 = yn1;
yn1 = yn;
}
// Apply a small bipolar impulse to filter to prevent arithmetic underflow.
yn1 += (synth_float_t) 1.0E-26;
yn2 -= (synth_float_t) 1.0E-26;
}
private:
synth_float_t mQ;
synth_float_t xn1; // delay lines
synth_float_t xn2;
double yn1;
double yn2;
synth_float_t a0; // coefficients
synth_float_t a1;
synth_float_t a2;
synth_float_t b1;
synth_float_t b2;
synth_float_t cos_omega;
synth_float_t sin_omega;
synth_float_t alpha;
// Calculate coefficients common to many parametric biquad filters.
void calcCommon( synth_float_t ratio, synth_float_t Q )
{
synth_float_t omega;
/* Don't let frequency get too close to Nyquist or filter will blow up. */
if( ratio >= 0.499f ) ratio = 0.499f;
omega = 2.0f * (synth_float_t)M_PI * ratio;
#if 1
// This is not significantly faster on Mac or Linux.
cos_omega = SynthTools::fastCosine(omega);
sin_omega = SynthTools::fastSine(omega );
#else
{
float fsin_omega;
float fcos_omega;
sincosf(omega, &fsin_omega, &fcos_omega);
cos_omega = (synth_float_t) fcos_omega;
sin_omega = (synth_float_t) fsin_omega;
}
#endif
alpha = sin_omega / (2.0f * Q);
}
// Lowpass coefficients
void calculateCoefficients( synth_float_t frequency, synth_float_t Q )
{
synth_float_t scalar, omc;
if( frequency < BIQUAD_MIN_FREQ ) frequency = BIQUAD_MIN_FREQ;
calcCommon( frequency * mSamplePeriod, Q );
scalar = 1.0f / (1.0f + alpha);
omc = (1.0f - cos_omega);
a0 = omc * 0.5f * scalar;
a1 = omc * scalar;
a2 = a0;
b1 = -2.0f * cos_omega * scalar;
b2 = (1.0f - alpha) * scalar;
}
};
};
#endif // SYNTHMARK_BIQUAD_FILTER_H
@@ -0,0 +1,78 @@
/*
* Copyright (C) 2016 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
* This code was translated from the JSyn Java code.
* JSyn is Copyright 2009 Phil Burk, Mobileer Inc
* JSyn is licensed under the Apache License, Version 2.0
*/
#ifndef SYNTHMARK_DIFFERENTIATED_PARABOLA_H
#define SYNTHMARK_DIFFERENTIATED_PARABOLA_H
#include <cstdint>
#include <math.h>
#include "SynthTools.h"
namespace marksynth {
constexpr double kDPWVeryLowFrequency = 2.0 * 0.1 / kSynthmarkSampleRate;
/**
* DPW is a tool for generating band-limited waveforms
* based on a paper by Antti Huovilainen and Vesa Valimaki:
* "New Approaches to Digital Subtractive Synthesis"
*/
class DifferentiatedParabola
{
public:
DifferentiatedParabola()
: mZ1(0)
, mZ2(0) {}
virtual ~DifferentiatedParabola() = default;
synth_float_t next(synth_float_t phase, synth_float_t phaseIncrement) {
synth_float_t dpw;
synth_float_t positivePhaseIncrement = (phaseIncrement < 0.0)
? phaseIncrement
: 0.0 - phaseIncrement;
// If the frequency is very low then just use the raw sawtooth.
// This avoids divide by zero problems and scaling problems.
if (positivePhaseIncrement < kDPWVeryLowFrequency) {
dpw = phase;
} else {
// Calculate the parabola.
synth_float_t squared = phase * phase;
// Differentiate using a delayed value.
synth_float_t diffed = squared - mZ2;
// Delay line.
// TODO - Why Z2. Vesa's paper says use Z1?
mZ2 = mZ1;
mZ1 = squared;
// Calculate scaling
dpw = diffed * 0.25f / positivePhaseIncrement; // TODO extract and optimize
}
return dpw;
}
private:
synth_float_t mZ1;
synth_float_t mZ2;
};
};
#endif // SYNTHMARK_DIFFERENTIATED_PARABOLA_H
@@ -0,0 +1,229 @@
/*
* Copyright (C) 2016 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
* This code was translated from the JSyn Java code.
* JSyn is Copyright 2009 Phil Burk, Mobileer Inc
* JSyn is licensed under the Apache License, Version 2.0
*/
#ifndef SYNTHMARK_ENVELOPE_ADSR_H
#define SYNTHMARK_ENVELOPE_ADSR_H
#include <cstdint>
#include <math.h>
#include "SynthTools.h"
#include "UnitGenerator.h"
namespace marksynth {
/**
* Generate a contour that can be used to control amplitude or
* other parameters.
*/
class EnvelopeADSR : public UnitGenerator
{
public:
EnvelopeADSR()
: mAttack(0.05)
, mDecay(0.6)
, mSustainLevel(0.4)
, mRelease(2.5)
{}
virtual ~EnvelopeADSR() = default;
#define MIN_DURATION (1.0 / 100000.0)
enum State {
IDLE, ATTACKING, DECAYING, SUSTAINING, RELEASING
};
void setGate(bool gate) {
triggered = gate;
}
bool isIdle() {
return mState == State::IDLE;
}
/**
* Time in seconds for the falling stage to go from 0 dB to -90 dB. The decay stage will stop at
* the sustain level. But we calculate the time to fall to -90 dB so that the decay
* <em>rate</em> will be unaffected by the sustain level.
*/
void setDecayTime(synth_float_t time) {
mDecay = time;
}
synth_float_t getDecayTime() {
return mDecay;
}
/**
* Time in seconds for the rising stage of the envelope to go from 0.0 to 1.0. The attack is a
* linear ramp.
*/
void setAttackTime(synth_float_t time) {
mAttack = time;
}
synth_float_t getAttackTime() {
return mAttack;
}
void generate(int32_t numSamples) {
for (int i = 0; i < numSamples; i++) {
switch (mState) {
case IDLE:
for (; i < numSamples; i++) {
output[i] = mLevel;
if (triggered) {
startAttack();
break;
}
}
break;
case ATTACKING:
for (; i < numSamples; i++) {
// Increment first so we can render fast attacks.
mLevel += increment;
if (mLevel >= 1.0) {
mLevel = 1.0;
output[i] = mLevel;
startDecay();
break;
} else {
output[i] = mLevel;
if (!triggered) {
startRelease();
break;
}
}
}
break;
case DECAYING:
for (; i < numSamples; i++) {
output[i] = mLevel;
mLevel *= mScaler; // exponential decay
if (mLevel < kAmplitudeDb96) {
startIdle();
break;
} else if (!triggered) {
startRelease();
break;
} else if (mLevel < mSustainLevel) {
mLevel = mSustainLevel;
startSustain();
break;
}
}
break;
case SUSTAINING:
for (; i < numSamples; i++) {
mLevel = mSustainLevel;
output[i] = mLevel;
if (!triggered) {
startRelease();
break;
}
}
break;
case RELEASING:
for (; i < numSamples; i++) {
output[i] = mLevel;
mLevel *= mScaler; // exponential decay
if (triggered) {
startAttack();
break;
} else if (mLevel < kAmplitudeDb96) {
startIdle();
break;
}
}
break;
}
}
}
private:
void startIdle() {
mState = State::IDLE;
mLevel = 0.0;
}
void startAttack() {
if (mAttack < MIN_DURATION) {
mLevel = 1.0;
startDecay();
} else {
increment = mSamplePeriod / mAttack;
mState = State::ATTACKING;
}
}
void startDecay() {
double duration = mDecay;
if (duration < MIN_DURATION) {
startSustain();
} else {
mScaler = SynthTools::convertTimeToExponentialScaler(duration, mSampleRate);
mState = State::DECAYING;
}
}
void startSustain() {
mState = State::SUSTAINING;
}
void startRelease() {
double duration = mRelease;
if (duration < MIN_DURATION) {
duration = MIN_DURATION;
}
mScaler = SynthTools::convertTimeToExponentialScaler(duration, mSampleRate);
mState = State::RELEASING;
}
synth_float_t mAttack;
synth_float_t mDecay;
/**
* Level for the sustain stage. The envelope will hold here until the input goes to zero or
* less. This should be set between 0.0 and 1.0.
*/
synth_float_t mSustainLevel;
/**
* Time in seconds to go from 0 dB to -90 dB. This stage is triggered when the input goes to
* zero or less. The release stage will start from the sustain level. But we calculate the time
* to fall from full amplitude so that the release <em>rate</em> will be unaffected by the
* sustain level.
*/
synth_float_t mRelease;
State mState = State::IDLE;
synth_float_t mScaler = 1.0;
synth_float_t mLevel = 0.0;
synth_float_t increment = 0;
bool triggered = false;
};
};
#endif // SYNTHMARK_ENVELOPE_ADSR_H
@@ -0,0 +1,36 @@
/*
* Copyright (C) 2016 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
* This code was translated from the JSyn Java code.
* JSyn is Copyright 2009 Phil Burk, Mobileer Inc
* JSyn is licensed under the Apache License, Version 2.0
*/
#ifndef INCLUDE_ME_ONCE_H
#define INCLUDE_ME_ONCE_H
#include "UnitGenerator.h"
#include "PitchToFrequency.h"
namespace marksynth {
//synth statics
int32_t UnitGenerator::mSampleRate = kSynthmarkSampleRate;
synth_float_t UnitGenerator::mSamplePeriod = 1.0f / kSynthmarkSampleRate;
PowerOfTwoTable PitchToFrequency::mPowerTable(64);
};
#endif //INCLUDE_ME_ONCE_H
@@ -0,0 +1,72 @@
/*
* Copyright (C) 2016 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
* This code was translated from the JSyn Java code.
* JSyn is Copyright 2009 Phil Burk, Mobileer Inc
* JSyn is licensed under the Apache License, Version 2.0
*/
#ifndef SYNTHMARK_LOOKUP_TABLE_H
#define SYNTHMARK_LOOKUP_TABLE_H
#include <cstdint>
#include "SynthTools.h"
namespace marksynth {
class LookupTable {
public:
LookupTable(int32_t numEntries)
: mNumEntries(numEntries)
{}
virtual ~LookupTable() {
delete[] mTable;
}
void fillTable() {
// Add 2 guard points for interpolation and roundoff error.
int tableSize = mNumEntries + 2;
mTable = new float[tableSize];
// Fill the table with calculated values
float scale = 1.0f / mNumEntries;
for (int i = 0; i < tableSize; i++) {
float value = calculate(i * scale);
mTable[i] = value;
}
}
/**
* @param input normalized between 0.0 and 1.0
*/
float lookup(float input) {
float fractionalTableIndex = input * mNumEntries;
int32_t index = (int) floor(fractionalTableIndex);
float fraction = fractionalTableIndex - index;
float baseValue = mTable[index];
float value = baseValue
+ (fraction * (mTable[index + 1] - baseValue));
return value;
}
virtual float calculate(float input) = 0;
private:
int32_t mNumEntries;
synth_float_t *mTable;
};
};
#endif // SYNTHMARK_LOOKUP_TABLE_H
@@ -0,0 +1,104 @@
/*
* Copyright (C) 2016 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
* This code was translated from the JSyn Java code.
* JSyn is Copyright 2009 Phil Burk, Mobileer Inc
* JSyn is licensed under the Apache License, Version 2.0
*/
#ifndef SYNTHMARK_PITCH_TO_FREQUENCY_H
#define SYNTHMARK_PITCH_TO_FREQUENCY_H
#include <cstdint>
#include <math.h>
#include "SynthTools.h"
#include "LookupTable.h"
namespace marksynth {
constexpr int kSemitonesPerOctave = 12;
// Pitches are in semitones based on the MIDI standard.
constexpr int kPitchMiddleC = 60;
constexpr double kFrequencyMiddleC = 261.625549;
class PowerOfTwoTable : public LookupTable {
public:
PowerOfTwoTable(int32_t numEntries)
: LookupTable(numEntries)
{
fillTable();
}
virtual ~PowerOfTwoTable() {}
virtual float calculate(float input) override {
return powf(2.0f, input);
}
};
class PitchToFrequency
{
public:
PitchToFrequency() {}
virtual ~PitchToFrequency() {
}
static double convertPitchToFrequency(double pitch) {
double exponent = (pitch - kPitchMiddleC) * (1.0 / kSemitonesPerOctave);
return kFrequencyMiddleC * pow(2.0, exponent);
}
synth_float_t lookupPitchToFrequency(synth_float_t pitch) {
// Only calculate if input changed since last time.
if (pitch != lastInput) {
synth_float_t octavePitch = (pitch - kPitchMiddleC) * (1.0 / kSemitonesPerOctave);
int32_t octaveIndex = (int) floor(octavePitch);
synth_float_t fractionalOctave = octavePitch - octaveIndex;
// Do table lookup.
synth_float_t value = kFrequencyMiddleC * mPowerTable.lookup(fractionalOctave);
// Adjust for octave by multiplying by a power of 2. Allow for +/- 16 octaves;
const int32_t octaveOffset = 16;
synth_float_t octaveScaler = ((synth_float_t)(1 << (octaveIndex + octaveOffset)))
* (1.0 / (1 << octaveOffset));
value *= octaveScaler;
lastInput = pitch;
lastOutput = value;
}
return lastOutput;
}
/**
* @param pitches an array of fractional MIDI pitches
*/
void generate(const synth_float_t *pitches, synth_float_t *frequencies, int32_t count) {
for (int i = 0; i < count; i++) {
frequencies[i] = lookupPitchToFrequency(pitches[i]);
}
}
private:
static PowerOfTwoTable mPowerTable;
synth_float_t lastInput = kPitchMiddleC;
synth_float_t lastOutput = kFrequencyMiddleC;
};
};
#endif // SYNTHMARK_PITCH_TO_FREQUENCY_H
@@ -0,0 +1,80 @@
/*
* Copyright (C) 2016 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
* This code was translated from the JSyn Java code.
* JSyn is Copyright 2009 Phil Burk, Mobileer Inc
* JSyn is licensed under the Apache License, Version 2.0
*/
#ifndef SYNTHMARK_SAWTOOTH_OSCILLATOR_H
#define SYNTHMARK_SAWTOOTH_OSCILLATOR_H
#include <cstdint>
#include <math.h>
#include "SynthTools.h"
#include "UnitGenerator.h"
#include "DifferentiatedParabola.h"
namespace marksynth {
/**
* Simple phasor that can be used to implement other oscillators.
* Note that this is NON-bandlimited and should not be used
* directly as a sound source.
*/
class SawtoothOscillator : public UnitGenerator
{
public:
SawtoothOscillator()
: mPhase(0) {}
virtual ~SawtoothOscillator() = default;
void generate(synth_float_t frequency, int32_t numSamples) {
synth_float_t phase = mPhase;
synth_float_t phaseIncrement = 2.0 * frequency * mSamplePeriod;
for (int i = 0; i < numSamples; i++) {
output[i] = translatePhase(phase, phaseIncrement);
phase += phaseIncrement;
if (phase > 1.0) {
phase -= 2.0;
}
}
mPhase = phase;
}
void generate(synth_float_t *frequencies, int32_t numSamples) {
synth_float_t phase = mPhase;
for (int i = 0; i < numSamples; i++) {
synth_float_t phaseIncrement = 2.0 * frequencies[i] * mSamplePeriod;
output[i] = translatePhase(phase, phaseIncrement);
phase += phaseIncrement;
if (phase > 1.0) {
phase -= 2.0;
}
}
mPhase = phase;
}
virtual synth_float_t translatePhase(synth_float_t phase, synth_float_t phaseIncrement) {
(void) phaseIncrement;
return phase;
}
private:
synth_float_t mPhase; // between -1.0 and +1.0
};
};
#endif // SYNTHMARK_SAWTOOTH_OSCILLATOR_H
@@ -0,0 +1,53 @@
/*
* Copyright (C) 2016 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
* This code was translated from the JSyn Java code.
* JSyn is Copyright 2009 Phil Burk, Mobileer Inc
* JSyn is licensed under the Apache License, Version 2.0
*/
#ifndef SYNTHMARK_SAWTOOTH_OSCILLATOR_DPW_H
#define SYNTHMARK_SAWTOOTH_OSCILLATOR_DPW_H
#include <cstdint>
#include <math.h>
#include "SynthTools.h"
#include "DifferentiatedParabola.h"
namespace marksynth {
/**
* Band limited sawtooth oscillator.
* Suitable as a sound source.
*/
class SawtoothOscillatorDPW : public SawtoothOscillator
{
public:
SawtoothOscillatorDPW()
: SawtoothOscillator()
, dpw() {}
virtual ~SawtoothOscillatorDPW() = default;
virtual inline synth_float_t translatePhase(synth_float_t phase, synth_float_t phaseIncrement) {
return dpw.next(phase, phaseIncrement);
}
private:
DifferentiatedParabola dpw;
};
};
#endif // SYNTHMARK_SAWTOOTH_OSCILLATOR_DPW_H
@@ -0,0 +1,137 @@
/*
* Copyright (C) 2016 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#ifndef SYNTHMARK_SIMPLE_VOICE_H
#define SYNTHMARK_SIMPLE_VOICE_H
#include <cstdint>
#include <math.h>
#include "SynthTools.h"
#include "VoiceBase.h"
#include "SawtoothOscillator.h"
#include "SawtoothOscillatorDPW.h"
#include "SquareOscillatorDPW.h"
#include "SineOscillator.h"
#include "EnvelopeADSR.h"
#include "PitchToFrequency.h"
#include "BiquadFilter.h"
namespace marksynth {
/**
* Classic subtractive synthesizer voice with
* 2 LFOs, 2 audio oscillators, filter and envelopes.
*/
class SimpleVoice : public VoiceBase
{
public:
SimpleVoice()
: VoiceBase()
, mLfo1()
, mOsc1()
, mOsc2()
, mPitchToFrequency()
, mFilter()
, mFilterEnvelope()
, mAmplitudeEnvelope()
// The following values are arbitrary but typical values.
, mDetune(1.0001f) // slight phasing
, mVibratoDepth(0.03f)
, mVibratoRate(6.0f)
, mFilterEnvDepth(3000.0f)
, mFilterCutoff(400.0f)
{
mFilter.setQ(2.0);
// Randomize attack times to smooth out CPU load for envelope state transitions.
mFilterEnvelope.setAttackTime(0.05 + (0.2 * SynthTools::nextRandomDouble()));
mFilterEnvelope.setDecayTime(7.0 + (1.0 * SynthTools::nextRandomDouble()));
mAmplitudeEnvelope.setAttackTime(0.02 + (0.05 * SynthTools::nextRandomDouble()));
mAmplitudeEnvelope.setDecayTime(1.0 + (0.2 * SynthTools::nextRandomDouble()));
}
virtual ~SimpleVoice() = default;
void setPitch(synth_float_t pitch) {
mPitch = pitch;
}
void noteOn(synth_float_t pitch, synth_float_t velocity) {
(void) velocity; // TODO use velocity?
mPitch = pitch;
mFilterEnvelope.setGate(true);
mAmplitudeEnvelope.setGate(true);
}
void noteOff() {
mFilterEnvelope.setGate(false);
mAmplitudeEnvelope.setGate(false);
}
void generate(int32_t numFrames) {
assert(numFrames <= kSynthmarkFramesPerRender);
// LFO #1 - vibrato
mLfo1.generate(mVibratoRate, numFrames);
synth_float_t *pitches = mBuffer1;
SynthTools::scaleOffsetBuffer(mLfo1.output, pitches, numFrames, mVibratoDepth, mPitch);
synth_float_t *frequencies = mBuffer2;
mPitchToFrequency.generate(pitches, frequencies, numFrames);
// OSC #1 - sawtooth
mOsc1.generate(frequencies, numFrames);
// OSC #2 - detuned square wave oscillator
SynthTools::scaleBuffer(frequencies, frequencies, numFrames, mDetune);
mOsc2.generate(frequencies, numFrames);
// Mix the two oscillators
synth_float_t *mixed = frequencies;
SynthTools::mixBuffers(mOsc1.output, 0.6, mOsc2.output, 0.4, mixed, numFrames);
// Filter envelope
mFilterEnvelope.generate(numFrames);
synth_float_t *cutoffFrequencies = pitches; // reuse unneeded buffer
SynthTools::scaleOffsetBuffer(mFilterEnvelope.output, cutoffFrequencies, numFrames,
mFilterEnvDepth, mFilterCutoff);
// Biquad resonant low-pass filter
mFilter.generate(mixed, cutoffFrequencies, numFrames);
// Amplitude ADSR
mAmplitudeEnvelope.generate(numFrames);
SynthTools::multiplyBuffers(mFilter.output, mAmplitudeEnvelope.output, output, numFrames);
}
private:
SineOscillator mLfo1;
SawtoothOscillatorDPW mOsc1;
SquareOscillatorDPW mOsc2;
PitchToFrequency mPitchToFrequency;
BiquadFilter mFilter;
EnvelopeADSR mFilterEnvelope;
EnvelopeADSR mAmplitudeEnvelope;
synth_float_t mDetune; // frequency scaler
synth_float_t mVibratoDepth; // in semitones
synth_float_t mVibratoRate; // in Hertz
synth_float_t mFilterEnvDepth; // in Hertz
synth_float_t mFilterCutoff; // in Hertz
// Buffers for storing signals that are being passed between units.
synth_float_t mBuffer1[kSynthmarkFramesPerRender];
synth_float_t mBuffer2[kSynthmarkFramesPerRender];
};
};
#endif // SYNTHMARK_SIMPLE_VOICE_H
@@ -0,0 +1,44 @@
/*
* Copyright (C) 2016 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
* This code was translated from the JSyn Java code.
* JSyn is Copyright 2009 Phil Burk, Mobileer Inc
* JSyn is licensed under the Apache License, Version 2.0
*/
#ifndef SYNTHMARK_SINE_OSCILLATOR_H
#define SYNTHMARK_SINE_OSCILLATOR_H
#include <cstdint>
#include <math.h>
#include "SynthTools.h"
namespace marksynth {
class SineOscillator : public SawtoothOscillator
{
public:
SineOscillator()
: SawtoothOscillator() {}
virtual ~SineOscillator() = default;
virtual inline synth_float_t translatePhase(synth_float_t phase, synth_float_t phaseIncrement) {
(void) phaseIncrement;
return SynthTools::fastSine(phase * M_PI);
}
};
};
#endif // SYNTHMARK_SINE_OSCILLATOR_H

Some files were not shown because too many files have changed in this diff Show More