The Wayback Machine - https://web.archive.org/web/20200909221353/https://github.com/dotnet/spark/issues/31
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Write unit tests for SimpleWorker. #31

Open
imback82 opened this issue Nov 28, 2018 · 2 comments
Open

Write unit tests for SimpleWorker. #31

imback82 opened this issue Nov 28, 2018 · 2 comments

Comments

@imback82
Copy link
Contributor

@imback82 imback82 commented Nov 28, 2018

SimpleWorker unit tests are missing in Spark.Net.Work.UnitTest.

@rapoth rapoth transferred this issue from another repository Apr 24, 2019
@KimKiHyuk
Copy link
Contributor

@KimKiHyuk KimKiHyuk commented Aug 21, 2020

i'm on it

@KimKiHyuk
Copy link
Contributor

@KimKiHyuk KimKiHyuk commented Aug 23, 2020

I did pre-requisite work following https://github.com/dotnet/spark/blob/master/docs/building/ubuntu-instructions.md

and then tried to run SimpleWorker but it couldn't be run because of

internal static int GetWorkerFactoryPort(Version version)
{
string portStr = (version >= new Version(Versions.V2_3_1)) ?
GetEnvironmentVariable("PYTHON_WORKER_FACTORY_PORT") :
Console.ReadLine();

GetEnvironmentVariable("PYTHON_WORKER_FACTORY_PORT") returns null value and it's used for port value

so I've changed SimpleWorker.cs

from

int port = Utils.SettingUtils.GetWorkerFactoryPort(_version);
string secret = Utils.SettingUtils.GetWorkerFactorySecret(_version);

to

string secret = Utils.SettingUtils.GetWorkerFactorySecret(_version);
ISocketWrapper clientSocket = SocketFactory.CreateSocket();
var ipEndpoint = (IPEndPoint)clientSocket.LocalEndPoint;
int port = ipEndpoint.Port;

clientSocket.Connect(IPAddress.Loopback, port, secret);
new TaskRunner(0, clientSocket, false, _version).Run();

you can see it KimKiHyuk@a1d1226

and it works properly.

test method is here

using System;
using System.Collections.Generic;
using System.Net;
using System.Threading.Tasks;
using Microsoft.Spark.Network;
using Xunit;

namespace Microsoft.Spark.Worker.UnitTest
{
    public class SimpleWorkerTests
    {
        [Fact]
        public void TestsSimpleWorkerTaskRunners()
        {
            var typedVersion = new Version(Versions.V2_4_0);
            var simpleWorker = new SimpleWorker(typedVersion);
            
            Task.Run(() => simpleWorker.Run());

            // mock Assert, change condition
            Assert.False(false);
        }
    }
}

do you think master branch code is wrong? or my setting is wrong?

actually, I couldn't find what's the purpose of PYTHON_WORKER_FACTORY_PORT and some background tasks about it

KimKiHyuk added a commit to KimKiHyuk/spark-1 that referenced this issue Aug 26, 2020
GetWorkerFactoryPort does not work properly, it caused connection
refused.

please see dotnet#31 (comment)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.