LIVY-866: Optimize Yarn GetApplications Query to prevent additional load on Yarn and Livy

[nileshrathi345]

What changes were proposed in this pull request?

Problem
Livy was calling YarnClient.getApplications(SPARK), which asks the ResourceManager for all Spark applications and then filters client-side by application tag. On clusters with many Spark apps, this produces large RM responses, adds unnecessary load on YARN, and can contribute to RM memory pressure (especially during session startup polling and leaked-app GC).

Solution
Use a tag-filtered GetApplicationsRequest so the RM returns only matching applications:

• getAppIdFromTag() — when resolving a session’s YARN app ID by tag, query with the lowercase session tag instead of scanning all Spark apps.
• leakedAppsGCThread — when cleaning up leaked apps, batch all pending leaked tags into one filtered request per GC cycle instead of fetching the full Spark app list.

Jira: https://issues.apache.org/jira/browse/LIVY-866

How was this patch tested?

Updated and ran SparkYarnAppSpec, including:

• "should get App Id" — verifies tag-based app ID lookup uses the filtered getApplications request.
• "should delete leak app when timeout" — verifies the leaked-app GC path uses the filtered request.

Was this patch authored or co-authored using generative AI tooling?

Yes, this was co-authored using Cursor to help understanding of solution for leakedAppsGCThread

[nileshrathi345]

@gyogal, @ArnavBalyan and @roczei could you please take a look when possible and review
thanks..!!!!

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 68.67%. Comparing base (0efa2d7) to head (d6dc967).
⚠️ Report is 31 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff              @@
##             master     #533      +/-   ##
============================================
+ Coverage     68.38%   68.67%   +0.29%     
- Complexity     1199     1214      +15     
============================================
  Files           106      106              
  Lines          6711     6793      +82     
  Branches        831      839       +8     
============================================
+ Hits           4589     4665      +76     
- Misses         1657     1661       +4     
- Partials        465      467       +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[roczei]

LGTM